# SQL Yodeler > Schema management and migrations for ClickHouse and Postgres. The schema is declared in TypeScript with chant's sql lexicon; the yodel CLI writes versioned migrations from it, lints them, and applies them behind an approval bound to a digest of the plan. Agents setting SQL Yodeler up in a repository: start with https://intentius.io/sql-yodeler/agents/. A person new to it: https://intentius.io/sql-yodeler/getting-started/. A page with a prompt carries it under "Optional: hand this page to your coding agent", and this file lists it under the page. Every prompt forbids applying to a shared environment, approving and merging; those stay with people. --- # Set up with a coding agent Source: https://intentius.io/sql-yodeler/agents/ ## Optional: hand this page to your coding agent ```text Set up SQL Yodeler in this repository. Read https://intentius.io/sql-yodeler/llms.txt first, then https://intentius.io/sql-yodeler/agents/ and follow it. Open a pull request with the result. Never run `yodel apply` against a shared environment, never run `chant approve`, `yodel approve` or `yodel override`, never edit the `chant/lifecycle` branch or `.chant/allowed_signers`, never merge; approvals and applies belong to people. ``` Setup needs no agent: [Your first migration](/sql-yodeler/getting-started/) and [Starting a project](/sql-yodeler/adoption/) give every step by hand. This page is for a coding agent adding SQL Yodeler to a repository, and for the person handing it the task. Run the agent in the repository and give it this prompt: ```text Set up SQL Yodeler in this repository. Read https://intentius.io/sql-yodeler/llms.txt first, then https://intentius.io/sql-yodeler/agents/ and follow it. Open a pull request with the result. Never run `yodel apply` against a shared environment, never run `chant approve`, `yodel approve` or `yodel override`, never edit the `chant/lifecycle` branch or `.chant/allowed_signers`, never merge; approvals and applies belong to people. ``` The last line is in every prompt on this site. Applying to a shared environment, approving a plan, changing the `chant/lifecycle` branch (where chant keeps approvals) and merging stay with people; the agent prepares the change and hands those steps over. So does overriding a policy rule with `yodel override`, which records a person's decision to apply a plan the rule denies, and editing `.chant/allowed_signers`, the signers file that decides whose sealed approvals count. ## What the agent reads | File | Holds | |---|---| | [`llms.txt`](https://intentius.io/sql-yodeler/llms.txt) | every page, with a one-line description and its prompt | | [`llms-full.txt`](https://intentius.io/sql-yodeler/llms-full.txt) | the text of every page in one file | | Copy page as Markdown, under each page's title | that page's text, with its prompt | A task page's own prompt sits under its title as "Optional: hand this page to your coding agent". The [Glossary](/sql-yodeler/glossary/) explains the chant terms the commands use. ## Steps for the agent 1. Check the repository: Node 22.12 or later, a git repository, and whether it already holds other code. The template's CI pipelines go in the project's `.github/workflows`, `.forgejo/workflows` and `.gitlab-ci.yml`, which a forge reads only at the repository root. If the repository holds other code, ask the user where the project goes. 2. Ask the user for the dialect (ClickHouse or Postgres), the database (ClickHouse) or schema (Postgres) the tables live in, and whether it exists already. 3. Make the project from the template with `yodel create`, from npm. It makes the project in a new directory, or one that holds nothing but `.git`, and refuses any other; check `git status` afterwards. ```sh npx @intentius/sql-yodeler@latest create --clickhouse --database --name npx @intentius/sql-yodeler@latest create --postgres --schema --name cd && npm install ``` 4. Edit `chant.config.ts`: each environment's default server address (`dev`, `prod`, and any the user names). Credentials stay out of files: the profiles name environment variables, and the template's README lists them. For another environment, add its profile, its entry in `yodel.config.ts`, and copies of `ops/migrate-dev.op.ts` and `ops/watch-dev.op.ts` with `dev` changed, then run `npm run ci` to render the pipelines again. 5. For a new database, declare the tables in `src/` (chant's `sql` lexicon; the template's `src/schema.ts` shows the form) and write the first migration with `npx yodel new init`. For a database that exists, do not write `src/` by hand: `npx yodel init --from --force` adopts it ([Starting a project](/sql-yodeler/adoption/)). It writes one row to that environment's history, so ask the user first and run it with the credentials they give you. 6. Run `npx yodel lint` and `npm run ci:check`. Both must pass. 7. Optionally, check the migrations on a local database: `npx yodel emulator up` starts one in Docker, and [Your first migration](/sql-yodeler/getting-started/) has the variables. `npx yodel plan dev` and `npx yodel apply dev` against it are safe; `yodel apply` stops at the approval, and that is where the agent stops too. 8. Commit the project (with `package-lock.json`) on a new branch and open a pull request. Check `git status` so the commit holds nothing else. 9. Tell the user what is left for them, from the template's README: create each environment's reader and writer database users, add the forge's secrets where [Setting up each forge](/sql-yodeler/forges/) says, approve each plan the pull request comment shows, and merge. ## Rules for the agent - `npx yodel --help` lists each command's options and exit codes; [the exit codes](/sql-yodeler/cli/#exit-codes) says what they mean. - Print the approve command a plan shows, and never run it. The approval is bound to that plan's digest, and it is the user's. - Never edit a migration that is committed on the default branch: write a new one with `npx yodel new `. `npx yodel lint` reports an edited one as a `checksum` error. - Print the `yodel override` command a plan shows for a denied policy rule, and never run it. An override is the user's decision, with the user's reason. - Never edit `.chant/allowed_signers`, the file that lists whose sealed approvals count. Adding or removing a signer is the user's change. - Silence a lint finding (`-- yodel:allow `) only with a reason the user gives. - Credentials go in the forge's secrets, never in a file or a commit. - Read a command's result from its `--json` output, never from its text. Each document has a JSON Schema; [JSON output](/sql-yodeler/cli/#json-output) lists them. ## Read-only tools (yodel mcp) An agent that should look at a project's state without a shell can use `yodel mcp`, a Model Context Protocol server on stdin and stdout. Register it with the agent's MCP client, started in the project with the reader's credentials in its environment (the ones `yodel plan` uses): ```json { "mcpServers": { "yodel": { "command": "npx", "args": ["yodel", "mcp"] } } } ``` `--dir ` serves a project in another directory. The tools are read-only: | Tool | Runs | Gives | |---|---|---| | `status` | `yodel status --json` | applied, pending, out-of-order and part-way migrations, checksum mismatches, the plan digest apply would ask approval for | | `plan` | `yodel plan --json` | what `yodel apply` would run; in `forPerson`, the approve command when the plan waits for approval, and `yodel override` for each policy rule that denies it | | `lint` | `yodel lint [--env ] --json` | the findings and silences; with `env`, against that environment's history | | `drift` | `yodel drift --json` | the declared objects changed out of band or gone | | `history` | `yodel status --json` | the migrations the history records applied: who, when, under which digest, and any repairs | | `explain` | `yodel lint --rules --json` | what a lint rule checks and whether it can be silenced (`topic`: the rule's id), or what an exit code means (`topic`: `0` to `5`) | Each tool answers with one envelope ([`mcp.schema.json`](https://intentius.io/sql-yodeler/schemas/v1/mcp.schema.json)): `schema` (the envelope's version, `1`), `tool`, `command` (the command line it ran), `exit` (its exit code, the same as on a terminal), `results` (what it printed with `--json`, or `null` when it failed), `resultsSchema` (the `$id` of the schema `results` follows), `error` (what it wrote to stderr) and `forPerson` (steps that belong to a person, each with the command the person runs). There is no tool that applies, approves, overrides a policy rule, repairs, rebases, writes a migration or touches `chant/lifecycle`. The server refuses to start with a tool named for `apply`, `approve`, `repair`, `rebase`, `new`, `init` or `override`, a tool that runs any command besides `status`, `plan`, `lint` and `drift`, or a tool not marked read-only, and no tool passes a flag that writes (`--replay`, `--update-checksum`, `--comment`, `--execute`). The commands in `forPerson` are the person's to run: the agent shows them and stops, as the never-line above says. --- # SQL Yodeler Source: https://intentius.io/sql-yodeler/ SQL Yodeler manages the schemas of ClickHouse and Postgres databases declaratively. You declare what the schema should be in TypeScript with chant's `sql` lexicon ([Declaring the schema](/sql-yodeler/schema/)), and the `yodel` CLI works out the change: it either applies it directly (the declarative path) or writes versioned migrations from it and applies those (the versioned path). Both paths apply behind an approval bound to a digest of the plan. Data migrations (rebuilds, backfills, expand-and-contract column changes) are steps inside a migration, recorded in the same history as the DDL, and resume where they stopped. | Page | What it covers | |---|---| | [Your first migration](/sql-yodeler/getting-started/) | from an empty directory to applied migrations on a local database: the template, the emulator, the variables, plan, approve and apply | | [Installing and configuring](/sql-yodeler/configuration/) | installing `yodel`, the project layout, `chant.config.ts` profiles, `yodel.config.ts`, the environment variables | | [Starting a project](/sql-yodeler/adoption/) | a new project from a starter template, `yodel init --from` to adopt a database that already exists, and `yodel init --baseline` for another environment that holds the same schema | | [Coming from another migration tool](/sql-yodeler/from-other-tools/) | for a database golang-migrate, goose, Flyway or dbmate manages: the concepts mapped, adopting it with `yodel init --from`, the old tool's files and table, and rollback with `yodel revert` | | [Declaring the schema](/sql-yodeler/schema/) | the schema as TypeScript in `src/`: one export per object in the database's own DDL, references with `${}`, splitting it across files, generating objects with code, composites, what the build checks and the editor | | [The two workflows](/sql-yodeler/workflows/) | the declarative path (`yodel plan`, `yodel apply` against the declared schema) and the versioned path (`yodel new`, then `yodel apply`) | | [Migrations](/sql-yodeler/migrations/) | the migrations directory, the history table, the apply lock, resume, out-of-order refusal, `yodel status`, `yodel rebase`, `yodel repair`, `yodel checkpoint`, and rollback with `yodel revert` | | [Data migrations](/sql-yodeler/steps/) | ClickHouse rebuilds, backfills, Postgres `PostgresMigrationOp` steps, `yodel cleanup` of what they keep, and manual steps | | [Access control](/sql-yodeler/access/) | Postgres row-level security, policies, roles and grants, and ClickHouse users, roles, row policies and grants: `access` per environment, what the schema owns and what the environment owns, the plan's access section, adopting them | | [Lint](/sql-yodeler/lint/) | the rules, `yodel:allow` silences, rule levels, `--env`, and the replay check (`--replay`) | | [Lint and plan on pull requests only](/sql-yodeler/lint-and-plan/) | `ci: { apply: false }`: lint, the replay check and the plan comment on every pull request with readers only, no apply pipeline, and turning apply on later | | [Setting up each forge](/sql-yodeler/forges/) | GitHub, GitLab and Forgejo side by side: the pipeline files, where the readers' and the writer's secrets go, the tokens for the plan comment, the drift watch and resuming, cloud roles over OIDC, runners and pull requests from forks | | [Approval](/sql-yodeler/approval/) | the migrations Op and its gate, the plan digest, the pull request comment, and what happens when the digest moves | | [Reports and the audit log](/sql-yodeler/audit/) | `yodel report`: every environment's migrations side by side, the audit log derived from the history and the gate ledger, its gaps, `audit.jsonl` and the HTML run view | | [Schema from an ORM](/sql-yodeler/orm/) | `sources` in `yodel.config.ts`: tables an ORM defines, read from the DDL it prints, planned and applied with the rest | | [Drift](/sql-yodeler/drift/) | `yodel drift` and the scheduled `WatchOp` | | [Webhook events](/sql-yodeler/notify/) | `notify` in `yodel.config.ts`: signed events for an apply waiting for approval, a refusal, a failure and drift, and checking them in a receiver | | [Generated schema reference](/sql-yodeler/schema-docs/) | `yodel docs`: an HTML reference and a Mermaid ERD generated from the declared schema | | [Topology](/sql-yodeler/topology/) | ClickHouse on a single node, a cluster, a `Replicated` database and ClickHouse Cloud | | [The commands](/sql-yodeler/cli/) | which command runs what (`yodel`, chant's CLI, the template's `just` targets), `yodel --help`, the exit codes, and the JSON Schemas of `--json` output | | [Set up with a coding agent](/sql-yodeler/agents/) | the prompt to hand a coding agent, the steps it follows to add SQL Yodeler to a repository, and what it leaves to people | | [Glossary](/sql-yodeler/glossary/) | the chant terms yodel's commands and output use, in yodel's terms | | [Claims status](/sql-yodeler/claims/) | every scenario claim, what it says, and whether it passed plain and was caught broken on ClickHouse and on Postgres, with the pages each one proves | Two example projects, one per dialect, show each of these on a real server. They are finished projects, and their READMEs are transcripts of the runs that made them, against chant's emulator, with the output each command printed. To follow along step by step, start with [Your first migration](/sql-yodeler/getting-started/): - `examples/clickhouse`: adoption, a migration, a sort-key rebuild, a backfill, a lint silence, an approval that stops holding, drift, the replay check. - `examples/postgres`: adoption, lint findings, a migration that fails part way and resumes, a column rename as an expand-and-contract step, a destructive change silenced, a fork resolved with `yodel rebase` and `yodel repair`, an out-of-order migration, drift, the replay check. The pages quote those runs. --- # Start a new schema Source: https://intentius.io/sql-yodeler/for/new-schema/ For a team starting a schema from nothing: declare it in TypeScript in a project from a starter template, and yodel writes the first migration from it on a local database. ## Works - A project with a profile, a migrate Op and a drift watch per environment, and pipelines for GitHub, GitLab and Forgejo - The first migration written from `src/schema.ts`, planned, approved and applied on chant's local emulator - Lint, the replay check and a plan comment on every pull request ## Differs ### By database #### ClickHouse - A project owns ClickHouse databases (`yodel create --clickhouse --database `). - A rebuild keeps the old table until `yodel cleanup` drops it, and lint flags mutations and rebuilds (`ch-mutation`, `ch-rebuild`). #### Postgres - A project owns Postgres schemas (`yodel create --postgres --schema `). - Roles stay the environment's: the schema declares the policies and grants that name them. ### By forge #### GitHub - The writer's secrets are secrets of a GitHub environment limited to `main`. - The plan comment and the drift issue use the job's own token. #### GitLab - The writer's variables are protected and scoped to the environment; the readers' are not protected. - The plan comment and the drift issue need `GITLAB_TOKEN`, and each drift watch runs from a pipeline schedule you create. #### Forgejo - Secrets belong to the repository, with no environments, so limit who can push branches. - Jobs need a runner with the `docker` label, and a job gets no OIDC token for cloud roles. ## First step #### ClickHouse ```sh npx @intentius/sql-yodeler@latest create my-schema --clickhouse --database events ``` #### Postgres ```sh npx @intentius/sql-yodeler@latest create my-schema --postgres --schema app ``` Next: [Your first migration](/sql-yodeler/getting-started/). ## Proof The scenario claims below run what this room relies on against a real server, once plain (it passes) and once with the behaviour broken (the claim catches it). [Claims status](/sql-yodeler/claims/) lists every claim. | Claim | What it says | Plain, broken | Last run | |---|---|---|---| | `new` | yodel new writes the next migration offline, with no dev database, and it applies; on Postgres, functions, procedures and triggers too, which lint checks | ClickHouse: pass, caught; Postgres: pass, caught | `c6f58a4`, 2026-10-10 | | `lint` | yodel lint --replay replays the migrations into a fresh database and each gives the schema it recorded; offline, yodel lint fails a migrations directory with a fork (exit 3), naming both migrations; a checkpoint replays alone to its recorded schema, and a fresh environment starts from it; yodel revert undoes the newest migration behind the gate, with a hand-written step for its data step, back to its parent's recorded schema, and refuses a checkpoint or a migration before one; yodel test runs a project's tests on databases replayed from the migrations, a seed meeting the backfill after it, fails a case whose assertion does not hold, and refuses an environment with a history; on Postgres, a unique index over rows with duplicates is refused before the gate by its generated pre-check (exit 4, naming the statement and the count), and yodel lint flags it as data-dependent | ClickHouse: pass, caught; Postgres: pass, caught | `c6f58a4`, 2026-10-10 | | `approval` | a pending migration applies only after chant approve of its plan digest, and the history records that digest; a failing pre-migration check refuses it, and so does a policy rule, read at the base commit, unless an override is recorded for that digest, for a yodel revert as for an apply; the audit log derived from the history and the ledger accounts for every approval and apply, and names one removed; a reader and a writer that are one user are refused, and the reader, its password a minted token, cannot write; an approval is refused once the migration changed after it, and nothing is applied | ClickHouse: pass, caught; Postgres: pass, caught | `c6f58a4`, 2026-10-10 | | `drift` | yodel drift reports a declared object changed out of band, naming the property, and one dropped; on the versioned path it compares with the newest applied migration's recorded schema, so a pending migration is not drift | ClickHouse: pass, caught; Postgres: pass, caught | `c6f58a4`, 2026-10-10 | ## Then read ### Tasks - [Your first migration](/sql-yodeler/getting-started/) - [Setting up each forge](/sql-yodeler/forges/) ### Background - [Declaring the schema](/sql-yodeler/schema/) - [The two workflows](/sql-yodeler/workflows/) - [Migrations](/sql-yodeler/migrations/) - [Approval](/sql-yodeler/approval/) ### Reference - [The commands](/sql-yodeler/cli/) - [Installing and configuring](/sql-yodeler/configuration/) - [Glossary](/sql-yodeler/glossary/) --- # Adopt a live database Source: https://intentius.io/sql-yodeler/for/adopt/ For a team whose database already exists: read it into declarations and a baseline migration, then change it only through migrations. ## Works - `yodel init --from ` reads the live database into `src/` and plans it back to no change before it writes anything - A baseline migration recorded as applied, with none of its statements run - Access adopted with the tables when the environment manages it: policies and grants, and on ClickHouse users and roles - `yodel init --baseline` for another environment that holds the same schema ## Differs ### By database #### ClickHouse - A project owns ClickHouse databases (`yodel create --clickhouse --database `). - A rebuild keeps the old table until `yodel cleanup` drops it, and lint flags mutations and rebuilds (`ch-mutation`, `ch-rebuild`). #### Postgres - A project owns Postgres schemas (`yodel create --postgres --schema `). - Roles stay the environment's: the schema declares the policies and grants that name them. ### By forge #### GitHub - The writer's secrets are secrets of a GitHub environment limited to `main`. - The plan comment and the drift issue use the job's own token. #### GitLab - The writer's variables are protected and scoped to the environment; the readers' are not protected. - The plan comment and the drift issue need `GITLAB_TOKEN`, and each drift watch runs from a pipeline schedule you create. #### Forgejo - Secrets belong to the repository, with no environments, so limit who can push branches. - Jobs need a runner with the `docker` label, and a job gets no OIDC token for cloud roles. ## First step ```sh YODEL_CREDENTIALS=writer npx yodel init --from --force ``` Next: [Adopting an existing database](/sql-yodeler/adoption/#adopting-an-existing-database-yodel-init-from). ## Proof The scenario claims below run what this room relies on against a real server, once plain (it passes) and once with the behaviour broken (the claim catches it). [Claims status](/sql-yodeler/claims/) lists every claim. | Claim | What it says | Plain, broken | Last run | |---|---|---|---| | `adopt` | yodel init --from adopts a live database without touching it, and yodel plan then shows no change; on Postgres its policies, row-level security and grants too, on ClickHouse its dictionaries, functions, roles, users, row policies and grants; yodel init --baseline records the baseline, behind the gate, in a second environment that holds the same schema, and refuses one that differs | ClickHouse: pass, caught; Postgres: pass, caught | `9329873`, 2026-10-10 | ## Then read ### Tasks - [Starting a project](/sql-yodeler/adoption/) - [Setting up each forge](/sql-yodeler/forges/) ### Background - [The two workflows](/sql-yodeler/workflows/) - [Access control](/sql-yodeler/access/#adopting-a-database) ### Reference - [The commands](/sql-yodeler/cli/) - [Installing and configuring](/sql-yodeler/configuration/) --- # Lint and plan on pull requests Source: https://intentius.io/sql-yodeler/for/lint-and-plan/ For a team that wants every migration reviewed in its pull request before SQL Yodeler applies anything. ## Works - `yodel lint` and the replay check on every pull request, with no database credentials - A plan comment per environment, made with read-only credentials, and a write probe that fails the job if they can write - The drift watch on its schedule, with a tracking issue - No apply pipeline and no writer in CI until you turn apply on ## Differs ### By database #### ClickHouse - A project owns ClickHouse databases (`yodel create --clickhouse --database `). - A rebuild keeps the old table until `yodel cleanup` drops it, and lint flags mutations and rebuilds (`ch-mutation`, `ch-rebuild`). #### Postgres - A project owns Postgres schemas (`yodel create --postgres --schema `). - Roles stay the environment's: the schema declares the policies and grants that name them. ### By forge #### GitHub - The writer's secrets are secrets of a GitHub environment limited to `main`. - The plan comment and the drift issue use the job's own token. #### GitLab - The writer's variables are protected and scoped to the environment; the readers' are not protected. - The plan comment and the drift issue need `GITLAB_TOKEN`, and each drift watch runs from a pipeline schedule you create. #### Forgejo - Secrets belong to the repository, with no environments, so limit who can push branches. - Jobs need a runner with the `docker` label, and a job gets no OIDC token for cloud roles. ## First step Set `ci: { forges: ["github", "gitlab", "forgejo"], apply: false }` in `yodel.config.ts`, then render the pipelines: ```sh npm run ci ``` Next: [Lint and plan on pull requests only](/sql-yodeler/lint-and-plan/). ## Proof The scenario claims below run what this room relies on against a real server, once plain (it passes) and once with the behaviour broken (the claim catches it). [Claims status](/sql-yodeler/claims/) lists every claim. | Claim | What it says | Plain, broken | Last run | |---|---|---|---| | `lint` | yodel lint --replay replays the migrations into a fresh database and each gives the schema it recorded; offline, yodel lint fails a migrations directory with a fork (exit 3), naming both migrations; a checkpoint replays alone to its recorded schema, and a fresh environment starts from it; yodel revert undoes the newest migration behind the gate, with a hand-written step for its data step, back to its parent's recorded schema, and refuses a checkpoint or a migration before one; yodel test runs a project's tests on databases replayed from the migrations, a seed meeting the backfill after it, fails a case whose assertion does not hold, and refuses an environment with a history; on Postgres, a unique index over rows with duplicates is refused before the gate by its generated pre-check (exit 4, naming the statement and the count), and yodel lint flags it as data-dependent | ClickHouse: pass, caught; Postgres: pass, caught | `c6f58a4`, 2026-10-10 | | `pr-comment` | the pull request comment lists the pending migrations with each statement's class, and the digest the gate asks for | ClickHouse: pass, caught; Postgres: pass, caught | `c6f58a4`, 2026-10-10 | ## Then read ### Tasks - [Lint and plan on pull requests only](/sql-yodeler/lint-and-plan/) - [Setting up each forge](/sql-yodeler/forges/) ### Background - [Lint](/sql-yodeler/lint/) - [The pull request comment](/sql-yodeler/approval/#the-pull-request-comment) - [Drift](/sql-yodeler/drift/) ### Reference - [The commands](/sql-yodeler/cli/) - [The lint rules](/sql-yodeler/lint/#the-rules) --- # Coming from another tool Source: https://intentius.io/sql-yodeler/for/coming-from/ For a team whose database another migration tool manages today: the concepts mapped, then the database adopted with init --from. ## Works - Each concept of a numbered-files migration tool mapped to its SQL Yodeler counterpart - The database adopted as it is, with the old tool's history table left out - Rollback with `yodel revert`, planned from the recorded schema, behind the same approval - A migration that fails part way resumes where it stopped; one out of order is refused ## Differs ### By database #### ClickHouse - A project owns ClickHouse databases (`yodel create --clickhouse --database `). - A rebuild keeps the old table until `yodel cleanup` drops it, and lint flags mutations and rebuilds (`ch-mutation`, `ch-rebuild`). #### Postgres - A project owns Postgres schemas (`yodel create --postgres --schema `). - Roles stay the environment's: the schema declares the policies and grants that name them. ### By forge #### GitHub - The writer's secrets are secrets of a GitHub environment limited to `main`. - The plan comment and the drift issue use the job's own token. #### GitLab - The writer's variables are protected and scoped to the environment; the readers' are not protected. - The plan comment and the drift issue need `GITLAB_TOKEN`, and each drift watch runs from a pipeline schedule you create. #### Forgejo - Secrets belong to the repository, with no environments, so limit who can push branches. - Jobs need a runner with the `docker` label, and a job gets no OIDC token for cloud roles. ## First step ```sh YODEL_CREDENTIALS=writer npx yodel init --from --force ``` Next: [Coming from another migration tool](/sql-yodeler/from-other-tools/). ## Proof The scenario claims below run what this room relies on against a real server, once plain (it passes) and once with the behaviour broken (the claim catches it). [Claims status](/sql-yodeler/claims/) lists every claim. | Claim | What it says | Plain, broken | Last run | |---|---|---|---| | `adopt` | yodel init --from adopts a live database without touching it, and yodel plan then shows no change; on Postgres its policies, row-level security and grants too, on ClickHouse its dictionaries, functions, roles, users, row policies and grants; yodel init --baseline records the baseline, behind the gate, in a second environment that holds the same schema, and refuses one that differs | ClickHouse: pass, caught; Postgres: pass, caught | `9329873`, 2026-10-10 | | `resume` | an interrupted apply resumes where it stopped: a backfill step written as yodel's form (table, key, batch size, SQL) from its receipts, running each batch once, and a failed statement at that statement, never resending one that ran | ClickHouse: pass, caught; Postgres: pass, caught | `c6f58a4`, 2026-10-10 | | `out-of-order` | a migration merged late, before one already applied, is refused unless --allow-out-of-order, and never skipped | ClickHouse: pass, caught; Postgres: pass, caught | `c6f58a4`, 2026-10-10 | ## Then read ### Tasks - [Coming from another migration tool](/sql-yodeler/from-other-tools/) - [Cutting CI over](/sql-yodeler/from-other-tools/#cutting-ci-over) - [Setting up each forge](/sql-yodeler/forges/) ### Background - [Migrations](/sql-yodeler/migrations/) - [Data migrations](/sql-yodeler/steps/) ### Reference - [The commands](/sql-yodeler/cli/) - [Glossary](/sql-yodeler/glossary/) --- # Setting up CI for many teams Source: https://intentius.io/sql-yodeler/for/platform/ For a platform team that runs the pipelines of many schema projects: one renderer, the same jobs on each forge, and a pinned image. ## Works - `yodel ci` renders every project's pipelines from the installed package, and `yodel ci --check` fails a pipeline edited by hand or not rendered again - `ci.forges` picks GitHub, GitLab or Forgejo, and `ci.jobs` keeps a project's own jobs across renders - `ci.image` runs every job in the CI image, pinned by its digest - Waves: one environment after another, each behind its own gate ## Differs ### By database #### ClickHouse - A project owns ClickHouse databases (`yodel create --clickhouse --database `). - A rebuild keeps the old table until `yodel cleanup` drops it, and lint flags mutations and rebuilds (`ch-mutation`, `ch-rebuild`). #### Postgres - A project owns Postgres schemas (`yodel create --postgres --schema `). - Roles stay the environment's: the schema declares the policies and grants that name them. ### By forge #### GitHub - The writer's secrets are secrets of a GitHub environment limited to `main`. - The plan comment and the drift issue use the job's own token. #### GitLab - The writer's variables are protected and scoped to the environment; the readers' are not protected. - The plan comment and the drift issue need `GITLAB_TOKEN`, and each drift watch runs from a pipeline schedule you create. #### Forgejo - Secrets belong to the repository, with no environments, so limit who can push branches. - Jobs need a runner with the `docker` label, and a job gets no OIDC token for cloud roles. ## First step ```sh npm run ci ``` Next: [The pipelines: yodel ci](/sql-yodeler/workflows/#the-pipelines-yodel-ci). ## Proof The scenario claims below run what this room relies on against a real server, once plain (it passes) and once with the behaviour broken (the claim catches it). [Claims status](/sql-yodeler/claims/) lists every claim. | Claim | What it says | Plain, broken | Last run | |---|---|---|---| | `waves` | the apply pipeline runs one wave per environment, in order, each behind its gate policy read from the base commit, applies only what the wave before applied, and applies a tenant set's migrations to every tenant behind one gate; a sealed wave counts only an approval sealed by a signer the base commit lists | ClickHouse: pass, caught; Postgres: pass, caught | `c6f58a4`, 2026-10-10 | | `template` | a project from the starter template, on Forgejo: apply only after approval, lint with replay and the plan comment on a pull request, and the approved change applied on merge; a sealed wave applies only on an approval sealed by a signer listed at the base, and a pr-review wave on the review of a writer other than the author; a pull request job cannot write, a forked migration fails lint and is annotated, a stale or hand-edited pipeline fails yodel ci --check, the CI image pinned by digest runs a pull request's jobs, a command token source mints the reader's password, and the drift watch keeps one tracking issue | ClickHouse: pass, caught; Postgres: pass, caught | `868ff97`, 2026-10-10 | ## Then read ### Tasks - [Setting up each forge](/sql-yodeler/forges/) - [Lint and plan on pull requests only](/sql-yodeler/lint-and-plan/) ### Background - [The pipelines: yodel ci](/sql-yodeler/workflows/#the-pipelines-yodel-ci) - [Waves: a gate per environment](/sql-yodeler/approval/#waves-a-gate-per-environment) - [Least privilege on each forge](/sql-yodeler/configuration/#least-privilege-on-each-forge) ### Reference - [yodel.config.ts](/sql-yodeler/configuration/#yodelconfigts) - [The commands](/sql-yodeler/cli/) --- # Working with a coding agent Source: https://intentius.io/sql-yodeler/for/agents/ For someone who hands SQL Yodeler's setup to a coding agent, and keeps approvals and applies for people. ## Works - One prompt that sets SQL Yodeler up in a repository and opens a pull request, and stops short of applying, approving or merging - `yodel mcp`: read-only tools for status, plan, lint and drift - JSON Schemas for the `--json` output of each command - llms.txt and a prompt on each task page ## Differs ### By database #### ClickHouse - A project owns ClickHouse databases (`yodel create --clickhouse --database `). - A rebuild keeps the old table until `yodel cleanup` drops it, and lint flags mutations and rebuilds (`ch-mutation`, `ch-rebuild`). #### Postgres - A project owns Postgres schemas (`yodel create --postgres --schema `). - Roles stay the environment's: the schema declares the policies and grants that name them. ### By forge #### GitHub - The writer's secrets are secrets of a GitHub environment limited to `main`. - The plan comment and the drift issue use the job's own token. #### GitLab - The writer's variables are protected and scoped to the environment; the readers' are not protected. - The plan comment and the drift issue need `GITLAB_TOKEN`, and each drift watch runs from a pipeline schedule you create. #### Forgejo - Secrets belong to the repository, with no environments, so limit who can push branches. - Jobs need a runner with the `docker` label, and a job gets no OIDC token for cloud roles. ## First step Hand the agent this prompt: ```text Set up SQL Yodeler in this repository. Read https://intentius.io/sql-yodeler/llms.txt first, then https://intentius.io/sql-yodeler/agents/ and follow it. Open a pull request with the result. Never run `yodel apply` against a shared environment, never run `chant approve`, `yodel approve` or `yodel override`, never edit the `chant/lifecycle` branch or `.chant/allowed_signers`, never merge; approvals and applies belong to people. ``` Next: [Set up with a coding agent](/sql-yodeler/agents/). ## Proof No scenario claim runs what this room describes. [Claims status](/sql-yodeler/claims/) lists every claim. ## Then read ### Tasks - [Set up with a coding agent](/sql-yodeler/agents/) - [Your first migration](/sql-yodeler/getting-started/) ### Background - [Rules for the agent](/sql-yodeler/agents/#rules-for-the-agent) - [Approval](/sql-yodeler/approval/) ### Reference - [Read-only tools (yodel mcp)](/sql-yodeler/agents/#read-only-tools-yodel-mcp) - [JSON output](/sql-yodeler/cli/#json-output) - [Exit codes](/sql-yodeler/cli/#exit-codes) --- # Security review Source: https://intentius.io/sql-yodeler/for/security/ What each job can reach, who can approve an apply, and the record every change leaves. ## Works - A reader and a writer per environment: only the apply wave holds the writer, and a write probe fails a pull request job that can write - Each approval bound to a digest of the plan; when the plan moves, the approval no longer holds - Approval modes, including approvals sealed with a signer listed in the repository - An audit log derived from the history and the gate ledger ## Differs ### By database #### ClickHouse - A project owns ClickHouse databases (`yodel create --clickhouse --database `). - A rebuild keeps the old table until `yodel cleanup` drops it, and lint flags mutations and rebuilds (`ch-mutation`, `ch-rebuild`). #### Postgres - A project owns Postgres schemas (`yodel create --postgres --schema `). - Roles stay the environment's: the schema declares the policies and grants that name them. ### By forge #### GitHub - The writer's secrets are secrets of a GitHub environment limited to `main`. - The plan comment and the drift issue use the job's own token. #### GitLab - The writer's variables are protected and scoped to the environment; the readers' are not protected. - The plan comment and the drift issue need `GITLAB_TOKEN`, and each drift watch runs from a pipeline schedule you create. #### Forgejo - Secrets belong to the repository, with no environments, so limit who can push branches. - Jobs need a runner with the `docker` label, and a job gets no OIDC token for cloud roles. ## First step ```sh npx yodel config check --write-probe ``` Next: [Credentials: a reader and a writer](/sql-yodeler/configuration/#credentials-a-reader-and-a-writer). ## Proof The scenario claims below run what this room relies on against a real server, once plain (it passes) and once with the behaviour broken (the claim catches it). [Claims status](/sql-yodeler/claims/) lists every claim. | Claim | What it says | Plain, broken | Last run | |---|---|---|---| | `approval` | a pending migration applies only after chant approve of its plan digest, and the history records that digest; a failing pre-migration check refuses it, and so does a policy rule, read at the base commit, unless an override is recorded for that digest, for a yodel revert as for an apply; the audit log derived from the history and the ledger accounts for every approval and apply, and names one removed; a reader and a writer that are one user are refused, and the reader, its password a minted token, cannot write; an approval is refused once the migration changed after it, and nothing is applied | ClickHouse: pass, caught; Postgres: pass, caught | `c6f58a4`, 2026-10-10 | | `waves` | the apply pipeline runs one wave per environment, in order, each behind its gate policy read from the base commit, applies only what the wave before applied, and applies a tenant set's migrations to every tenant behind one gate; a sealed wave counts only an approval sealed by a signer the base commit lists | ClickHouse: pass, caught; Postgres: pass, caught | `c6f58a4`, 2026-10-10 | ## Then read ### Tasks - [Setting up each forge](/sql-yodeler/forges/) - [Access control](/sql-yodeler/access/) ### Background - [Approval modes](/sql-yodeler/approval/#approval-modes-which-approvals-count) - [Sealed approvals](/sql-yodeler/approval/#sealed-approvals) - [The plan digest](/sql-yodeler/approval/#the-plan-digest) - [The audit log](/sql-yodeler/audit/#the-audit-log) ### Reference - [Least privilege on each forge](/sql-yodeler/configuration/#least-privilege-on-each-forge) - [yodel config check](/sql-yodeler/configuration/#yodel-config-check) - [Claims status](/sql-yodeler/claims/) --- # Evaluating SQL Yodeler Source: https://intentius.io/sql-yodeler/for/evaluate/ For someone deciding whether SQL Yodeler fits: two finished example projects, the claims that run against real servers, and a first migration on a local database. ## Works - Two example projects, one per dialect, whose READMEs are transcripts of real runs - Scenario claims that run each feature against a real server, plain and with the behaviour broken - Your first migration on chant's local emulator, with no cloud account ## Differs ### By database #### ClickHouse - A project owns ClickHouse databases (`yodel create --clickhouse --database `). - A rebuild keeps the old table until `yodel cleanup` drops it, and lint flags mutations and rebuilds (`ch-mutation`, `ch-rebuild`). #### Postgres - A project owns Postgres schemas (`yodel create --postgres --schema `). - Roles stay the environment's: the schema declares the policies and grants that name them. ### By forge #### GitHub - The writer's secrets are secrets of a GitHub environment limited to `main`. - The plan comment and the drift issue use the job's own token. #### GitLab - The writer's variables are protected and scoped to the environment; the readers' are not protected. - The plan comment and the drift issue need `GITLAB_TOKEN`, and each drift watch runs from a pipeline schedule you create. #### Forgejo - Secrets belong to the repository, with no environments, so limit who can push branches. - Jobs need a runner with the `docker` label, and a job gets no OIDC token for cloud roles. ## First step #### ClickHouse ```sh npx @intentius/sql-yodeler@latest create my-schema --clickhouse --database events ``` #### Postgres ```sh npx @intentius/sql-yodeler@latest create my-schema --postgres --schema app ``` Next: [Your first migration](/sql-yodeler/getting-started/). ## Proof The scenario claims below run what this room relies on against a real server, once plain (it passes) and once with the behaviour broken (the claim catches it). [Claims status](/sql-yodeler/claims/) lists every claim. | Claim | What it says | Plain, broken | Last run | |---|---|---|---| | `new` | yodel new writes the next migration offline, with no dev database, and it applies; on Postgres, functions, procedures and triggers too, which lint checks | ClickHouse: pass, caught; Postgres: pass, caught | `c6f58a4`, 2026-10-10 | | `template` | a project from the starter template, on Forgejo: apply only after approval, lint with replay and the plan comment on a pull request, and the approved change applied on merge; a sealed wave applies only on an approval sealed by a signer listed at the base, and a pr-review wave on the review of a writer other than the author; a pull request job cannot write, a forked migration fails lint and is annotated, a stale or hand-edited pipeline fails yodel ci --check, the CI image pinned by digest runs a pull request's jobs, a command token source mints the reader's password, and the drift watch keeps one tracking issue | ClickHouse: pass, caught; Postgres: pass, caught | `868ff97`, 2026-10-10 | | `template-github` | a project from the starter template, on GitHub Actions (act and a mock GitHub): apply only after approval, lint with replay and the plan comment on a pull request, and the approved change applied on merge; a sealed wave applies only on an approval sealed by a signer listed at the base, and a pr-review wave on the review of a writer other than the author; a pull request job cannot write, a forked migration fails lint and is annotated, a stale or hand-edited pipeline fails yodel ci --check, the CI image pinned by digest runs a pull request's jobs, a command token source mints the reader's password, and the drift watch keeps one tracking issue | ClickHouse: pass, caught; Postgres: pass, caught | `868ff97`, 2026-10-10 | ## Then read ### Tasks - [Your first migration](/sql-yodeler/getting-started/) ### Background - [The ClickHouse example](https://github.com/INTENTIUS/sql-yodeler/tree/main/examples/clickhouse) - [The Postgres example](https://github.com/INTENTIUS/sql-yodeler/tree/main/examples/postgres) - [The two workflows](/sql-yodeler/workflows/) ### Reference - [Claims status](/sql-yodeler/claims/) - [The commands](/sql-yodeler/cli/) --- # Declaring the schema Source: https://intentius.io/sql-yodeler/schema/ SQL Yodeler is declarative: `src/` says what the database should be, and yodel works out the change from what it is. The schema is TypeScript in `src/`. Each exported constant is one object: a database or schema, a table, a view, an index. Its body is the database's own DDL, in a tagged template from chant's `sql` lexicon, so a statement reads the way `SHOW CREATE` prints it. TypeScript holds the statements together: objects import each other, refer to each other with `${}`, and composites make families of them. `yodel new` writes the next migration from the change since the last one, and `yodel plan` shows a declarative change against the live server ([The two workflows](/sql-yodeler/workflows/)). #### ClickHouse ```ts // src/schema.ts import { database, table } from "@intentius/chant-lexicon-sql/clickhouse"; export const db = database` CREATE DATABASE events ENGINE = Atomic`; export const events = table` CREATE TABLE ${db}.events ( id UInt64, kind LowCardinality(String), at DateTime ) ENGINE = MergeTree ORDER BY (kind, at)`; ``` #### Postgres ```ts // src/schema.ts import { index, schema, table } from "@intentius/chant-lexicon-sql/postgres"; export const app = schema`CREATE SCHEMA app`; export const orders = table` CREATE TABLE ${app}.orders ( id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY, status text NOT NULL DEFAULT 'placed', amount numeric(12,2) NOT NULL, placed_at timestamptz NOT NULL DEFAULT now() )`; export const ordersByStatus = index` CREATE INDEX orders_status_idx ON ${orders} (${orders.columns.status})`; ``` The export name is the object's identity, and the name in the SQL is its name on the server. Keep the export name when you change the name in the SQL, and the change is planned as a rename, not as a drop and a create. ## Schema as data The build reads `src/` as data: it reduces each file to the objects it declares, without running it. So the same source gives the same schema on any machine and in every environment, the schema at two commits can be compared with no database, and nothing in it depends on where or when it was built. That makes `src/` a small part of TypeScript: exported objects, constants and strings, imports between files, `${}` references, and composites called with literal options. What a program would do is not schema: - Reading an environment variable or a file, or querying a database. The schema is the same in every environment; what differs between them, the server and the credentials, is in `chant.config.ts`'s profiles ([Installing and configuring](/sql-yodeler/configuration/)). - `if` or a loop around exports, `let`, and functions or callbacks in `src/`. Write each object as an export, or put the repetition in a composite ([Generating objects](#generating-objects)). A file outside that part still builds, but chant runs it instead of reading it, and `chant build --verbose` names the file and the reason. chant's [TypeScript as data](https://intentius.io/chant/concepts/typescript-as-data/) page has the whole subset. ## References An interpolated object is a reference, not text. `${db}.events` still reads `events.events` in the statement sent to the server, and the build also knows the table belongs to that database, so it creates the database first. A column is referenced through `.columns`: ```ts // src/views.ts import { view } from "@intentius/chant-lexicon-sql/clickhouse"; import { db, events } from "./schema.js"; export const byKind = view` CREATE VIEW ${db}.events_by_kind AS SELECT ${events.columns.kind} AS kind, count() AS n FROM ${events} GROUP BY kind`; ``` A misspelled reference, `${events.columns.knd}`, fails the build and says which interpolation is undefined. When a view reads every column through a reference, the build also records which columns each of its output columns comes from. A name written as plain text is SQL, not a reference: `FROM events.events` builds and runs, but the build cannot order the view after the table. Plain text is the way to name something the project does not declare, such as a table another team owns (`REFERENCES billing.accounts (id)`) or a role the environment keeps. ## Splitting the schema Any number of files in `src/` make up the schema, and they import each other like any TypeScript modules: a database in `src/db.ts`, a table per file, the views beside the tables they read. The build orders the objects by their references, whatever the file order. Plain `.sql` files in `src/` and DDL an ORM prints join the same build ([Schema from an ORM](/sql-yodeler/orm/)). ## Generating objects A family of objects with the same shape is a composite of your own: the shape once, in a module outside `src/`, and one call per object in `src/`. A rollup table per region: ```ts // lib/rollup.ts import { Composite } from "@intentius/chant/composite"; import { table, type ClickHouseDatabase } from "@intentius/chant-lexicon-sql/clickhouse"; export const DailyRollup = Composite((props: { db: ClickHouseDatabase; region: string }) => ({ table: table` CREATE TABLE ${props.db}.${`daily_${props.region}`} ( day Date, kind LowCardinality(String), n UInt64 ) ENGINE = SummingMergeTree ORDER BY (day, kind)`, }), "DailyRollup"); ``` ```ts // src/daily.ts import { DailyRollup } from "../lib/rollup.js"; import { db } from "./schema.js"; export const { table: dailyEu } = DailyRollup({ db, region: "eu" }); export const { table: dailyUs } = DailyRollup({ db, region: "us" }); export const { table: dailyApac } = DailyRollup({ db, region: "apac" }); ``` The build reads `src/daily.ts` as data and writes three `CREATE TABLE` statements. A fourth region is one more line, and `yodel new` writes its table into the next migration. ## Composites The `sql` lexicon ships composites of its own, for shapes many schemas need. On Postgres, `TenantTable` puts the tenant column first in the table's primary key and in an index: ```ts // src/tickets.ts import { TenantTable } from "@intentius/chant-lexicon-sql/postgres"; import { app } from "./schema.js"; export const { table: tickets, index: ticketsByTenant } = TenantTable({ name: "tickets", schema: app, columns: "id bigint GENERATED ALWAYS AS IDENTITY, title text NOT NULL, created_at timestamptz NOT NULL DEFAULT now()", primaryKey: "id", indexOn: "created_at DESC", }); ``` That builds a table with `PRIMARY KEY (tenant_id, id)` and the index `tickets_tenant_idx` on `(tenant_id, created_at DESC)`. The others: `SoftDeleteTable`, `AuditLogTable`, `JoinTable` and `RefreshedView` on Postgres; `EventsTable`, `ReplacingTable`, `RollupView`, `CdcMirror` and `ShardedTable` on ClickHouse. chant's [composites page](https://intentius.io/chant/lexicons/sql/composites/) has each one's options. ## Data changes The declaration says what the schema should be; some changes also move data. When the change from the last migration is one no statement makes in place, `yodel new` writes it as a data step of the same migration: a ClickHouse sort-key or engine change as a rebuild into a new table, a Postgres column rename or type change as expand and contract, and a backfill you write with `yodel new --backfill`. A data step runs under the same lock and approval as the statements, and resumes where it stopped ([Data migrations](/sql-yodeler/steps/)). ## What the build checks `chant build`, which yodel runs itself before it writes or plans a migration, parses every statement and checks it against the catalog of a pinned server, read from that release's own system tables: ClickHouse 26.8, and each Postgres major from 14 to 18. A mistake fails the build with a message that names the object, the column and the release: - a column type the server does not have, in any spelling it does not accept (`UInt46`, `uint64`, `numerik`), wrong type parameters (`Decimal(100, 2)`, `numeric(2000,2)`, `bigint(8)`), or a nesting it refuses (`Nullable(Array(String))`) - an engine, codec, codec parameter, MergeTree setting, Postgres storage parameter, index access method or operator class the server does not have - a function the server does not have, in a default, a key, a partition, a TTL, a check or an index - a column named in a key, an index, a foreign key or a grant that the table does not declare, and a foreign key between columns whose types cannot compare - a column declared twice, or two exports that declare the same object Names written as plain text for objects the project does not declare, and column names inside expressions, are left to the server. Lint rules then flag what the server would accept but should not: a primary key that is not a prefix of the sort key, a partition finer than a day, a timestamp without time zone, a foreign key no index leads with. [Lint](/sql-yodeler/lint/) has yodel's own rules on migrations, and chant's [lint rules page](https://intentius.io/chant/lexicons/sql/lint-rules/) has the build's. ## In the editor chant's language server (`chant serve lsp`, set up as [chant's LSP page](https://intentius.io/chant/cli/lsp/) shows) completes inside the templates from the same catalog: engines after `ENGINE =`, column types, codecs, settings and functions, and after `${` the tables and views the project declares and their columns. Hovering a type, an engine or a `${}` reference shows what it is, and chant's lint findings appear as you type. ## Proven by The scenario claims below run what this page describes against a real server, once plain (it passes) and once with the behaviour broken (the claim catches it). [Claims status](/sql-yodeler/claims/) lists every claim. | Claim | What it says | Plain, broken | Last run | |---|---|---|---| | `new` | yodel new writes the next migration offline, with no dev database, and it applies; on Postgres, functions, procedures and triggers too, which lint checks | ClickHouse: pass, caught; Postgres: pass, caught | `c6f58a4`, 2026-10-10 | | `declarative` | the declarative path: yodel plan shows the change against the live server and yodel apply makes it behind the plan-bound gate, for src/ declarations (on Postgres, functions, procedures and triggers, and access control: a role, a policy and grants, with a grant made by hand revoked; on ClickHouse, a dictionary, a function, and access control: a role, a user, a row policy and grants, with a grant made by hand revoked) and for an ORM's exported DDL | ClickHouse: pass, caught; Postgres: pass, caught | `c6f58a4`, 2026-10-10 | --- # Data migrations Source: https://intentius.io/sql-yodeler/steps/ ## Optional: hand this page to your coding agent ```text Write the data change I describe as a step of a migration, following https://intentius.io/sql-yodeler/steps/. For a backfill, run `npx yodel new --backfill`, fill in `backfills/.json` (the table, the key, the batch size and the SQL of one batch between `{from}` and `{to}`), and run the same command again. For a Postgres column rename, write `-- previously: ` on the new column's line in src/ before `npx yodel new `. Run `npx yodel lint`, and open a pull request with src/, backfills/ and the new migration. Never run `yodel apply` against a shared environment, never run `chant approve`, `yodel approve` or `yodel override`, never edit the `chant/lifecycle` branch or `.chant/allowed_signers`, never merge; approvals and applies belong to people. ``` A migration changes data as well as schema. Most of a migration is statements; a change that is not one statement, such as a backfill, a ClickHouse rebuild or a Postgres rename that readers must survive, is a data step of the migration, behind the same approval and recorded in the same history as the DDL: `yodel new` writes it into `migration.json` with the Op that makes it, and into `migration.sql` as a comment. `yodel apply` runs three kinds of Op step itself, in their place among the statements (the statements before a step first, the ones after it only once it succeeded), under the same lock and behind the same approval: | Step | Dialect | Written by `yodel new` for | |---|---|---| | `ClickHouseRebuildOp` | ClickHouse | a change `ALTER` cannot make: the sort key, the engine, a key column's type or name | | backfill | ClickHouse, Postgres | `yodel new --backfill`: a data migration you write, as a table, a key, a batch size and the SQL of one batch | | `PostgresMigrationOp` | Postgres | a column rename (SQLPG205) or a type change across kinds (SQLPG208) | A step and any file it runs are in the migration's checksum, so in the plan digest the approval binds. Its history rows have `kind = 'step'`, and the `succeeded` (or `failed`) row's `note` is the step's summary as JSON. A step that failed or was stopped runs again on the next `yodel apply` and resumes from its receipts, skipping the partitions or batches that finished. `yodel lint --replay` runs rebuild and `PostgresMigrationOp` steps in the replayed database and skips backfills, which move data the replayed database does not have. Anything else, a manual step or an Op step `yodel apply` does not run (a rebuild in app mode), is refused before anything runs: `yodel plan` and `yodel status` exit 2, `yodel apply` exits 2 and applies nothing, and `yodel lint` reports it (rule `refused-step`). The output quoted here is from the examples' runs; `...` marks lines left out. ## ClickHouse rebuilds Changing the sort key of the ClickHouse example's `events` table, `ORDER BY (kind, at)` to `ORDER BY (kind, id)`: ```console $ npx yodel new events-by-id Wrote migrations/20261010T1722-events-by-id/ (0 statements; follows 20261010T1722-add-country) This migration contains an Op step: ClickHouseRebuildOp for events (shop.events), SQLCH220. It is not SQL. migration.json holds the Op's declaration: export const { op } = ClickHouseRebuildOp({ name: "rebuild-shop-events", env: "", table: "shop.events", dualWrite: { mode: "materialized-view", cutoverColumn: "at" } }); ``` ```sql -- yodel migration 20261010T1722-events-by-id -- parent: 20261010T1722-add-country -- This migration contains an Op step, not only SQL: -- ClickHouseRebuildOp for events (shop.events), SQLCH220: not SQL; migration.json holds its declaration -- yodel:allow ch-rebuild 600 rows; the copy takes seconds -- events (shop.events): SQLCH220 made by ClickHouseRebuildOp, not a statement: -- export const { op } = ClickHouseRebuildOp({ name: "rebuild-shop-events", env: "", table: "shop.events", dualWrite: { mode: "materialized-view", cutoverColumn: "at" } }); ``` (The `yodel:allow` line was added after `yodel lint` flagged the rebuild; see [Lint](/sql-yodeler/lint/#silencing-a-finding).) The step is chant's `ClickHouseRebuildOp`, run by chant's local executor inside the apply, with the migration's recorded schema as its declaration (the table is rebuilt as this migration declares it, however far `src/` has moved since). In materialized-view mode it: 1. creates `__chant_new` with the new definition; 2. creates a materialized view, `
__chant_dual`, that writes every row inserted at or after a cut-over time (now plus a minute, by the table's time column, `cutoverColumn`) into the new table; 3. waits for the cut-over, then copies the table partition by partition, writing a receipt for each partition: the rows from before the cut-over, then the rows at or after it (future timestamps) that the view has not written, so every row the table held is copied once; 4. verifies that counts and checksums of every row, per partition, match in both tables, and fails the step with nothing swapped if they do not; 5. compares the two tables again just before the swap, and fails with nothing swapped if a row reached only the old table since the verification; otherwise swaps the tables (`EXCHANGE TABLES`), drops the view, and keeps the old table as `
__chant_old` until a retention date 7 days on. [`yodel cleanup`](#cleaning-up-what-steps-kept) drops it after that date. yodel builds the Op with `gates: "outer"`, so it has no gate and no Drop phase of its own: its approval is the migration's. The replay check runs the same step in an empty database, which shows the statements it sends: ```console $ npx yodel lint --replay replay --reset --reset-shared dev ... replay: 20261010T1722-events-by-id: replaying -- shop.events needs a rebuild (SQLCH220 Change the sorting key); 4 column(s) copied -- SQLCH220 orderBy: ( kind , at ) -> ( kind , id ) CREATE TABLE `shop`.`events__chant_new` ( id UInt64, kind LowCardinality(String), at DateTime, country LowCardinality(String) DEFAULT '' ) ENGINE = MergeTree PARTITION BY toYYYYMM(at) ORDER BY (kind, id) COMMENT '[chant managed-by=chant rebuild=shop.events role=new]' CREATE MATERIALIZED VIEW `shop`.`events__chant_dual` TO `shop`.`events__chant_new` AS SELECT `id`, `kind`, `at`, `country` FROM `shop`.`events` WHERE `at` >= toDateTime64('2026-10-10 17:23:41.000', 3, 'UTC') COMMENT 'chant rebuild of shop.events: rows at or after the cut-over [chant managed-by=chant rebuild=shop.events role=dual cutover=2026-10-10T17%3A23%3A41.000Z]' -- waiting 6s for the cut-over at 2026-10-10T17:23:41.000Z -- backfill of shop.events: 0 partition(s), 0 copied, 0 already copied, 0 cleared first -- shop.events: 0 partition(s), 0 row(s) (0 of them before the cut-over at 2026-10-10T17:23:41.000Z), counts and checksums equal in the old and new tables EXCHANGE TABLES `shop`.`events` AND `shop`.`events__chant_new` DROP VIEW `shop`.`events__chant_dual` SYNC RENAME TABLE `shop`.`events__chant_new` TO `shop`.`events__chant_old` ALTER TABLE `shop`.`events__chant_old` MODIFY COMMENT 'chant rebuild of shop.events: the old table, kept until 2026-10-17T17:23:41.114Z [chant managed-by=chant rebuild=shop.events role=old retain-until=2026-10-17T17%3A23%3A41.114Z]' ALTER TABLE `shop`.`events` MODIFY COMMENT '[chant managed-by=chant]' replay: 20261010T1722-events-by-id: matches its recorded schema ... ``` In the example's apply, with 600 rows in six monthly partitions: ```console $ npx yodel apply dev ... Running migrate-dev (/clickhouse/ops/migrate-dev.op.ts) lock: no KeeperMap on this server (KeeperMap is disabled because 'keeper_map_path_prefix' config is not defined. (BAD_ARGUMENTS) (version 26.8.15.10 (official build))); the lock is a file on this machine, so cross-runner locking needs Keeper (and keeper_map_path_prefix) approval: migrate-dev / approve-migrate-dev for jcs1-sha256:82c892052401d2b5a35d49964395124736586c6a8222858f8f0905155d29eb47, by yodel at 2026-10-10T17:22:36.240Z 20261010T1722-events-by-id: applying step 0: ClickHouseRebuildOp for events (shop.events) step 0: ok {"op":"ClickHouseRebuildOp","table":"shop.events","state":"rebuild","partitions":6,"copied":6,"skipped":0,"cleared":0,"repaired":0,"verifiedRows":600,"swapped":true,"oldTable":"shop.events__chant_old","retainUntil":"2026-10-17T17:22:47.304Z"} 20261010T1722-events-by-id: applied ... ``` The Op is also built with `onFailure: "keep"`, so a rebuild that failed or was killed keeps its new table and resumes on the next apply from the backfill's receipts: every partition with a receipt is skipped. To start it over instead, drop `
__chant_new` and `
__chant_dual` (a refusal or a failed verification names them). On a cluster of more than one shard the rebuild copies and verifies every shard; see [Topology](/sql-yodeler/topology/). `yodel new` picks materialized-view mode when the table has a time column to cut over on. A rebuild in app mode (the application writes both tables) is refused by `yodel apply`. The topology the rebuild renders for is the environment's, the same as the statements around it ([Topology](/sql-yodeler/topology/)). `yodel lint` reports every rebuild as `ch-rebuild`, an error unless silenced or lowered ([Lint](/sql-yodeler/lint/)). ## Backfills A backfill is a data migration in the same history as the DDL: "fill this column, a range of ids at a time". You write it as a form of a few fields, and yodel runs it as batches, each with its own receipt, so an apply that stops part way resumes at the first batch that has none. ### Writing one `yodel new --backfill` writes a template at `backfills/.json` when there is none, and no migration: ```json { "table": ".
", "key": "id", "batchSize": 100000, "settings": { "mutations_sync": "2" }, "sql": "write the backfill of fill-country: ALTER TABLE .
UPDATE = WHERE id >= {from} AND id < {to}" } ``` Fill it in. Filling `events.country` in the ClickHouse example's schema, 100,000 ids a batch: ```json { "table": "shop.events", "key": "id", "batchSize": 100000, "settings": { "mutations_sync": "2" }, "sql": "ALTER TABLE shop.events UPDATE country = ['DE', 'FR', 'US'][id % 3 + 1] WHERE id >= {from} AND id < {to} AND country = ''" } ``` Run the same command again. `yodel new` checks the form, writes it into the migration's step in `migration.json` (so it counts in the checksum and the plan digest), and ends the migration with the step. With `--backfill`, a migration is written even when the declared schema has not changed. It prints: ```text Wrote migrations/20261010T1212-fill-country/ (0 statements; follows 20261010T1212-baseline) This migration ends with a backfill of shop.events by id, 100000 a batch. Each batch runs: ALTER TABLE shop.events UPDATE country = ['DE', 'FR', 'US'][id % 3 + 1] WHERE id >= {from} AND id < {to} AND country = ''; yodel apply runs the batches after the statements before it, each with its own receipt, and resumes from them after a failure. ``` and `migration.sql` shows the step as comments: ```sql -- yodel migration 20261010T1212-fill-country -- parent: 20261010T1212-baseline -- This migration contains an Op step, not only SQL: -- a backfill of shop.events by id, 100000 a batch, its batches resumable from their receipts -- backfill (fill-country): a backfill of shop.events by id, 100000 a batch, not a statement; each batch runs: -- ALTER TABLE shop.events UPDATE country = ['DE', 'FR', 'US'][id % 3 + 1] WHERE id >= {from} AND id < {to} AND country = ''; ``` `yodel plan` lists the step with the table, the batches and the SQL of one batch, and the pull request comment shows the same under "Op steps", with the form as written. `--backfill ` reads the form from another `.json` file in the project. ### The form | Field | | |---|---| | `table` | the table the backfill fills, qualified as in your SQL | | `key`, `batchSize` | batches by ranges of an integer column: each range is `batchSize` wide, `{from}` inclusive to `{to}` exclusive | | `batches` | instead of `key` and `batchSize`: the batches as a list, strings or numbers, each one `{batch}` in the SQL | | `sql` | the SQL of one batch, an `UPDATE` or an `INSERT ... SELECT`: one statement, or a list run in order | | `settings` | optional: ClickHouse query settings for each statement (`mutations_sync: "2"` makes an `ALTER TABLE ... UPDATE` finish before the batch's receipt is written); on Postgres, settings for the batch's transaction (`work_mem`) | With `key`, the ranges come from the table's smallest and largest key when the step runs, aligned to multiples of `batchSize`, so a rerun sees the same ranges and skips the ones with a receipt. Rows added since the first run that fall in a range already done are not filled again; filter on the value still being empty (as `country = ''` does) and run the backfill as a new migration if that matters. A key that is not an integer (a date, a string) is refused when the step runs; list the batches with `batches` instead, for example one per month: ```json { "table": "shop.events", "batches": ["202606", "202607", "202608"], "settings": { "mutations_sync": "2" }, "sql": "ALTER TABLE shop.events UPDATE country = ['DE', 'FR', 'US'][id % 3 + 1] WHERE toYYYYMM(at) = {batch} AND country = ''" } ``` `{batch}` is replaced as written, so quote it in the SQL when it is a string (`'{batch}'`). ### How it runs `yodel apply` runs the step after the statements before it, under the migration's lock and approval. For each batch it reads the batch's receipt, skips the batch when the receipt is there, and otherwise runs the batch's statements and writes the receipt last, only when they all succeeded. A run that fails or is stopped leaves the finished batches' receipts, and the next `yodel apply` resumes at the first batch without one. The step's history note counts the batches: `{"op":"backfill","table":"shop.events","name":"backfill-fill-country","effects":3,"skipped":2,"ran":1}` is a rerun that found two batches done and ran the third. On ClickHouse, make each batch safe to run twice: the batch that was running when a run stopped runs again on the next one. The example's `UPDATE` touches only rows still empty. On Postgres each batch is one transaction with its receipt, so a batch is never half done ([Backfills on Postgres](#backfills-on-postgres)). The receipts are rows on the environment's server, addressed `yodel///`, so the same batch in two migrations is two receipts. They are kept by the sql lexicon's receipt store (`sqlReceiptStore` from `@intentius/chant-lexicon-sql/receipts`). On ClickHouse they are in `chant_receipts.receipts`, the table the rebuild keeps its partition receipts in. ### Backfills on Postgres On Postgres the batches' SQL runs on the apply's own connection to the environment's server, and the receipts are rows in `.__chant_receipts` (chant's Postgres receipts table, the one `PostgresMigrationOp` keeps in the migrated table's schema), in the schema yodel's history is in: `yodeler.__chant_receipts` by default. Each batch runs in one transaction: `BEGIN` at its first statement, then its statements, then its receipt, then `COMMIT`. A batch whose statement fails is rolled back whole and leaves no receipt, and a run killed in the middle of a batch leaves neither its changes nor its receipt, so the next `yodel apply` runs that batch again from its first statement and skips every batch that committed. A Postgres batch therefore does not need to be safe to run twice, but its statements must be ones Postgres runs in a transaction block: no `CREATE INDEX CONCURRENTLY`, no `VACUUM`. Keep batches small enough that holding their row locks until the batch commits is acceptable. The batch's transaction has `lock_timeout` and `statement_timeout` set from the environment's profile (`lockTimeoutMs`, default 5 s, and `scanTimeoutMs`, default none). A batch blocked behind another session's lock fails when the lock timeout passes; the next `yodel apply` resumes at it. The form's `settings` are set for the batch's transaction only (`set_config(name, value, true)`), for example `{ "work_mem": "256MB" }`. A Postgres backfill filling a new column, 10,000 ids a batch: ```json { "table": "shop.orders", "key": "id", "batchSize": 10000, "sql": "UPDATE shop.orders SET region = lower(country) WHERE id >= {from} AND id < {to}" } ``` An `INSERT ... SELECT` works the same way, for example copying rows into a new table a range at a time: `INSERT INTO shop.orders_archive SELECT * FROM shop.orders WHERE id >= {from} AND id < {to} AND placed_at < '2025-01-01'`. ### A backfill as a chant Op For a backfill the form cannot say (batches that are not a list or ranges of one key, a step that calls another activity, a check between batches), write the Op yourself. `yodel new --backfill ` with a `` that is not `.json` takes a module exporting a chant Op whose every step is an `effect()` batch with its own receipt, and writes a template at `` when it does not exist, and no migration: ```console $ npx yodel new fill-country --backfill backfills/country.ts Wrote a backfill template at backfills/country.ts. Write its batches, then run yodel new fill-country --backfill backfills/country.ts again. No migration written. ``` The ClickHouse example's backfill fills a column added two migrations earlier, one month per batch: ```ts /** * Fills events.country, one month of events per batch. Each batch is an * effect() with its own receipt, so a run that stops part way resumes at the * first month without one. The UPDATE touches only rows still empty, so a * batch that ran half way can run again. * * yodel new --backfill copies this file into the migration's directory as * backfill.ts; that copy is the one yodel apply runs, and it counts in the * migration's checksum. */ import { EffectReceipt } from "@intentius/chant"; import { Op, activity, effect, phase } from "@intentius/chant/op"; const months = ["202606", "202607", "202608", "202609", "202610", "202611"]; const batch = (month: string) => effect(EffectReceipt(`fill-country-${month}`, { effect: `fill-country/${month}`, flavor: "existence" }), [ activity( "yodelSql", { sql: `ALTER TABLE shop.events UPDATE country = ['DE', 'FR', 'US'][id % 3 + 1] WHERE toYYYYMM(at) = ${month} AND country = ''`, settings: { mutations_sync: "2" }, }, "atMostOnce", ), ]); export const op = Op({ name: "backfill-fill-country", overview: "events.country from the id, a month at a time", phases: [phase("Backfill", months.map(batch))], }); ``` Run again, `yodel new` copies the file into the migration's directory as `backfill.ts` and ends the migration with the step. With `--backfill`, a migration is written even when the declared schema has not changed. ```console $ npx yodel new fill-country --backfill backfills/country.ts Wrote migrations/20261010T1722-fill-country/ (0 statements; follows 20261010T1722-events-by-id) This migration ends with a backfill step: the Op backfill.ts exports, copied into the migration's directory. yodel apply runs its effect() batches after the statements before it, resuming from their receipts after a failure. ``` ```sql -- yodel migration 20261010T1722-fill-country -- parent: 20261010T1722-events-by-id -- This migration contains an Op step, not only SQL: -- a backfill: the Op backfill.ts exports, its effect() batches resumable from their receipts -- backfill (fill-country): made by the Op backfill.ts exports, not a statement ``` At apply time, chant's `effect()` cycle runs each batch: read its receipt, skip the batch if the receipt is there, otherwise run its steps and write the receipt last, only when they all succeeded. ```console $ npx yodel apply dev ... Running migrate-dev (/clickhouse/ops/migrate-dev.op.ts) lock: no KeeperMap on this server (KeeperMap is disabled because 'keeper_map_path_prefix' config is not defined. (BAD_ARGUMENTS) (version 26.8.15.10 (official build))); the lock is a file on this machine, so cross-runner locking needs Keeper (and keeper_map_path_prefix) approval: migrate-dev / approve-migrate-dev for jcs1-sha256:96f8f3625b2628eedd9a5567254b9b7ecf5e53ae0a22550d6da7cf3a37ff200e, by yodel at 2026-10-10T17:22:58.009Z 20261010T1722-fill-country: applying step 0: backfill backfill.ts ALTER TABLE shop.events UPDATE country = ['DE', 'FR', 'US'][id % 3 + 1] WHERE toYYYYMM(at) = 202606 AND country = '' ALTER TABLE shop.events UPDATE country = ['DE', 'FR', 'US'][id % 3 + 1] WHERE toYYYYMM(at) = 202607 AND country = '' ALTER TABLE shop.events UPDATE country = ['DE', 'FR', 'US'][id % 3 + 1] WHERE toYYYYMM(at) = 202608 AND country = '' ALTER TABLE shop.events UPDATE country = ['DE', 'FR', 'US'][id % 3 + 1] WHERE toYYYYMM(at) = 202609 AND country = '' ALTER TABLE shop.events UPDATE country = ['DE', 'FR', 'US'][id % 3 + 1] WHERE toYYYYMM(at) = 202610 AND country = '' ALTER TABLE shop.events UPDATE country = ['DE', 'FR', 'US'][id % 3 + 1] WHERE toYYYYMM(at) = 202611 AND country = '' step 0: ok {"op":"backfill","file":"backfill.ts","name":"backfill-fill-country","effects":6,"skipped":0,"ran":6} 20261010T1722-fill-country: applied ... ``` The rules for a backfill module: - Each step is an `effect()` batch. Its SQL runs with `activity("yodelSql", { sql, settings? })`, which yodel provides, on the environment's server; any activity chant has works too. - On ClickHouse, make each batch safe to run twice. A batch that was running when a run stopped runs again on the next one. The example's `UPDATE` touches only rows still empty. (On Postgres a batch is one transaction with its receipt; see [Backfills on Postgres](#backfills-on-postgres).) - Import packages only, no relative paths: the copy in the migration's directory is the one that runs. - No gates: the migration's approval covers it. - It counts in the migration's checksum, so editing `backfill.ts` after the migration ran is a checksum mismatch. The receipts are rows on the environment's server, addressed `yodel///`, so the same effect name in two migrations is two receipts. They are kept by the sql lexicon's receipt store (`sqlReceiptStore` from `@intentius/chant-lexicon-sql/receipts`). On ClickHouse they are in `chant_receipts.receipts`, the table the rebuild keeps its partition receipts in. The template `yodel new --backfill ` writes in a Postgres project raises an exception (`DO $$ BEGIN RAISE EXCEPTION ... END $$`) in place of the SQL until you write it. ## Postgres: PostgresMigrationOp Renaming a column in place breaks every reader still using the old name the moment it runs. `yodel new` cannot tell a rename from a dropped column and an added one unless the declaration says so: write `-- previously: ` on the new column's line. Without it, the Postgres example's rename came out as a manual step, with a hint (from a run made while writing the example; the migration was deleted after): ```console $ npx yodel new rename-email Wrote migrations/20261010T1724-rename-email/ (0 statements; follows 20261010T1724-refunds) This migration contains a manual step: orders (shop.orders), SQLPG203, SQLPG204. No statement and no Op makes it: shop.orders needs expand and contract, which no in-place statement makes, so nothing was sent for it. SQLPG203 Add a NOT NULL column with no default (columns.email - -> email text): ADD COLUMN ... NOT NULL with no default fails on a table that has rows. Add it nullable, backfill it, then set NOT NULL. https://www.postgresql.org/docs/18/sql-altertable.html No migration Op makes this change yet: make it by hand as expand and contract (add the new, write both, backfill, move readers, then drop the old). hint: orders: customer_email is dropped and email added with the same type in the same place; if it is a rename, write -- previously: customer_email on its line ``` With the line (`email text NOT NULL, -- previously: customer_email`), it is a `PostgresMigrationOp` step: ```console $ npx yodel new rename-email Wrote migrations/20261010T1724-rename-email/ (0 statements; follows 20261010T1724-refunds) This migration contains an Op step: PostgresMigrationOp for orders (shop.orders), SQLPG205. It is not SQL. migration.json holds the Op's declaration: export const { op } = PostgresMigrationOp({ name: "migrate-shop-orders-email", env: "", table: "shop.orders", column: "email" }); No retain: the old column is dropped in the same yodel apply, right after the switch, so anything still reading it breaks then. To keep it while readers move over: yodel new rename-email --replace --retain 7d ``` ```sql -- yodel migration 20261010T1724-rename-email -- parent: 20261010T1724-refunds -- This migration contains an Op step, not only SQL: -- PostgresMigrationOp for orders (shop.orders), SQLPG205: not SQL; migration.json holds its declaration -- orders (shop.orders): SQLPG205 made by PostgresMigrationOp, not a statement: -- export const { op } = PostgresMigrationOp({ name: "migrate-shop-orders-email", env: "", table: "shop.orders", column: "email" }); ``` The step is chant's expand-and-contract Op, built from the migration's recorded schema: Plan, Expand (add the new column), Dual write (a trigger keeps both columns written), Backfill (in batches, each receipt committed with its batch in `.__chant_receipts`), Carry over (indexes and constraints), Verify (every row's new column equals the expression over the old, and no NULL where NOT NULL is declared; a difference fails the step with nothing switched), Switch, Retain, Contract (drop the old column and the trigger, once its retention date has passed). yodel builds it with `gates: "outer"`, so the Op has neither of its two gates and the migration's approval covers the switch and the contract, and with `onFailure: "keep"`, so a failure drops nothing the step made. ```console $ npx yodel apply dev ... Running migrate-dev (/postgres/ops/migrate-dev.op.ts) lock: held in Postgres advisory lock (1498367052, 748638931) for history schema yodeler approval: migrate-dev / approve-migrate-dev for jcs1-sha256:8fceea63a48a1227fd5ddc01089e850a7c827140ad376098c33fca9eee489eec, by yodel at 2026-10-10T17:24:43.127Z 20261010T1724-rename-email: applying step 0: PostgresMigrationOp for orders (shop.orders) step 0: ok {"op":"PostgresMigrationOp","table":"shop.orders","column":"email","state":"migrate","change":"rename","filled":1,"skipped":0,"backfilledRows":301,"carried":0,"verifiedRows":301,"switched":true,"oldColumn":"shop.orders.customer_email","retainUntil":"2026-10-10T17:24:49.958Z","dropped":true} 20261010T1724-rename-email: applied ... ``` `retainUntil` and `dropped` in the note: by default the step keeps the old column for no time, and the contract drops it in the same apply, right after the switch, as the recorded schema says it is gone. Anything still reading `customer_email` breaks at that moment, which is what `yodel new` warns about. To keep the old column while readers move over, give the step a retention with `--retain`, when you write the migration or by writing it again before it is applied: ```sh npx yodel new rename-email --retain 7d # when writing it npx yodel new rename-email --replace --retain 7d # a migration written without it, not applied yet ``` `--retain` writes `"retain": "7d"` into the step's options in `migration.json` (and into its declaration), so it is in the checksum and the plan digest; never edit `migration.json` by hand to add it. Durations are whole numbers of `ms`, `s`, `m`, `h` or `d`. The lifecycle is then: 1. The apply expands, dual-writes, backfills, verifies and switches. Readers of `email` see the new column; the old column is still there under its old name, kept written by the dual-write trigger for a rename, and its comment says until when (`retain-until`). The step succeeds with `"dropped":false` and `retainUntil` in its note. 2. During those seven days, move the remaining readers and writers of `customer_email` over. 3. After the date, [`yodel cleanup `](#cleaning-up-what-steps-kept) drops the old column, and for a rename its trigger and function, behind an approval. Before the date it drops nothing. `--retain` sets the retention of a ClickHouse rebuild too: how long `
__chant_old` is kept after the swap, 7 days when the step sets none. A `PostgresMigrationOp` step that failed or was stopped keeps the new column, its trigger and the receipts of the batches it filled, and resumes on the next apply: each phase reads the server again and does what is left, and the backfill skips every batch with a receipt (the note's `skipped`). To start over instead, drop the new column, its trigger and function (named in their comments). The expand adds the new column at the end of the table, and no `ALTER` moves it, so the live column order differs from the declared one. `yodel drift` compares Postgres columns by name, so the order is not drift: declare the renamed column where the old one was. ## Cleaning up what steps kept Two steps keep something on purpose after they succeed, until a retention date written in its own comment (`retain-until` in chant's trailer): - a ClickHouse rebuild keeps the old table, `
__chant_old`, 7 days after the swap, or the step's `retain`; - a Postgres rename or type change keeps the old column after the switch when the step sets `retain`, and for a rename the trigger and function that keep it written. `yodel plan ` names them with their dates in a note, and so does the pull request comment. `yodel cleanup ` lists them, with the server's clock, and drops the ones whose date has passed. Nothing is dropped before its date. ```text $ npx yodel cleanup prod Kept by steps in prod (server time 2026-10-20T00:00:00.000Z): shop.events__chant_old (the old table of the rebuild of shop.events): kept until 2026-10-17T05:30:05.216Z, due shop.daily__chant_old (the old table of the rebuild of shop.daily): kept until 2026-10-27T00:00:00.000Z Plan digest: jcs1-sha256:... 1 due; nothing dropped without an approval. Approve it with: npx yodel approve prod --plan jcs1-sha256:... then run yodel cleanup prod again. ``` The drop is approved the way an apply is: the digest covers each due object by its identity on the server (a table by its UUID, a column by its table and attribute number) and its date, and the approval is of that digest on the migrations Op's cleanup gate: the Op's gate name with `-cleanup` (`approve-migrate-prod-cleanup`), a gate of its own, so an apply's approval and a cleanup's never stand in for each other, and `yodel plan` never names a cleanup's approval when it explains an apply's. With it, `yodel cleanup prod` takes the apply lock, reads the objects again, checks the approval against what it reads, and drops them: a ClickHouse table with `DROP TABLE ... SYNC` (rendered for the environment's topology), a Postgres column with its trigger and function in one transaction under the lock timeout, and the column migration's receipts after it. If anything due changed since the approval (another table came due, a table was rebuilt again), the digest differs and nothing is dropped. `--list` lists and never drops; `--json` prints the list, the digest and what was dropped. It exits 3 while something due waits for its approval, so a scheduled job can run it and fail visibly. The environment's approval mode counts as it does for an apply ([Approval](/sql-yodeler/approval/)): under `sealed`, only an approval sealed with `yodel approve --sign` by a key the signers file at the base commit lists for its approver drops anything, and the command `yodel cleanup` prints ends in `--sign`. ## Manual steps A change neither a statement nor an Op makes is a manual step: `yodel new` writes it and says what it needs, and `yodel apply` refuses the migration with exit 2 until it is gone. Make the change another way (for example, split it into changes yodel can make, as `-- previously:` does for a rename), delete the migration, and write it again. ## The ownership marker Objects yodel creates carry `[chant managed-by=chant]` at the end of their comment, on Postgres even a table that has no comment of its own (`COMMENT ON TABLE ... IS '[chant managed-by=chant]'`). chant, the library yodel declares and diffs schemas with, reads the marker to tell the objects it manages from ones it does not, and takes it off again before it compares or prints a comment, so your declared comment is what `yodel drift` and `yodel plan` compare. The working objects of a step carry more pairs in the same trailer: `
__chant_old` after a rebuild carries `role=old` and `retain-until=`, and so does the old column after a Postgres rename. Leave the marker in place: an object whose marker was removed by hand reads as one yodel did not create, until the next apply of its declaration stamps it again. ## Proven by The scenario claims below run what this page describes against a real server, once plain (it passes) and once with the behaviour broken (the claim catches it). [Claims status](/sql-yodeler/claims/) lists every claim. | Claim | What it says | Plain, broken | Last run | |---|---|---|---| | `rebuild` | ClickHouse: a sort-key change is a ClickHouseRebuildOp step inside the migration, never an ALTER or a drop; approved, it runs and keeps every row, and without an approval it does not run; yodel cleanup drops the old table it kept only after its retention date, behind an approval on a gate of its own, sealed under a sealed environment | ClickHouse: pass, caught | `c6f58a4`, 2026-10-10 | | `resume` | an interrupted apply resumes where it stopped: a backfill step written as yodel's form (table, key, batch size, SQL) from its receipts, running each batch once, and a failed statement at that statement, never resending one that ran | ClickHouse: pass, caught; Postgres: pass, caught | `c6f58a4`, 2026-10-10 | | `column-change` | Postgres: a column rename runs as a PostgresMigrationOp step (expand, backfill, switch, contract) and keeps every value; a step that fails part way keeps its work and resumes from its receipts | Postgres: pass, caught | `c6f58a4`, 2026-10-10 | --- # Your first migration Source: https://intentius.io/sql-yodeler/getting-started/ ## Optional: hand this page to your coding agent ```text Follow https://intentius.io/sql-yodeler/getting-started/ in a new directory: make the project, start the local emulator, and write and plan the first migration. Run `npx yodel apply dev` against the emulator, and when it stops at the approval, show me the approve command it printed and stop there. Never run `yodel apply` against a shared environment, never run `chant approve`, `yodel approve` or `yodel override`, never edit the `chant/lifecycle` branch or `.chant/allowed_signers`, never merge; approvals and applies belong to people. ``` This page takes you from an empty directory to two migrations applied to a ClickHouse or Postgres database on your own machine. It needs Node 22.12 or later, npm, git, and Docker for the local database. Every command below ran as shown, and the output under it is what it printed; `` stands for the project's path. ## What chant is SQL Yodeler is built on [chant](https://intentius.io/chant/) (`@intentius/chant` on npm), a declarative infrastructure-as-code toolkit in TypeScript. You declare your schema with chant's `sql` lexicon, and the `yodel` CLI does the rest, running chant's CLI underneath where it needs it: - `yodel create` makes a new project from a template. - `yodel emulator up` starts a local ClickHouse and Postgres in Docker. - `yodel approve ` approves a plan before `yodel apply` runs it: it shows the plan, asks you to type the environment's name, and records the approval for you. Before your project exists, run yodel as `npx @intentius/sql-yodeler@latest` (with `@latest`, npx does not reuse an older copy it cached). Inside the project, `npm install` puts the version the project pins in `node_modules`, so `npx yodel` works there. ## 1. Make the project #### ClickHouse ```sh npx @intentius/sql-yodeler@latest create my-schema --clickhouse --database events --name events-schema cd my-schema npm install ``` `--database events` is the ClickHouse database your schema lives in, and `--name` the package name (the directory's name when left out). The template declares one table in it, in `src/schema.ts`. This declaration is what the database should be; every migration below is worked out from it ([Declaring the schema](/sql-yodeler/schema/) has the form): ```ts export const db = database` CREATE DATABASE events ENGINE = Atomic COMMENT 'Declared in src/schema.ts'`; export const events = table` CREATE TABLE ${db}.events ( id UInt64, kind LowCardinality(String), at DateTime ) ENGINE = MergeTree ORDER BY (kind, at) COMMENT 'One row per event'`; ``` #### Postgres ```sh npx @intentius/sql-yodeler@latest create my-schema --postgres --schema app --name app-schema cd my-schema npm install ``` `--schema app` is the Postgres schema your objects live in. The template declares a table and an index in it, in `src/schema.ts`. This declaration is what the database should be; every migration below is worked out from it ([Declaring the schema](/sql-yodeler/schema/) has the form): ```ts export const app = schema` CREATE SCHEMA app; COMMENT ON SCHEMA app IS 'Declared in src/schema.ts'`; export const events = table` CREATE TABLE ${app}.events ( id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY, kind text NOT NULL, at timestamptz NOT NULL DEFAULT now() )`; export const eventsKind = index` CREATE INDEX events_kind_idx ON ${events} (${events.columns.kind})`; ``` `yodel` finds the Op that applies migrations through git, and chant keeps approvals on a git branch, so the project must be a git repository: ```sh git init -b main git add -A && git commit -m "chore: a new yodel project" ``` ## 2. Start a local database ```sh npx yodel emulator up ``` This starts ClickHouse on `http://127.0.0.1:8123` (user `default`, no password) and Postgres on `127.0.0.1:5432` (user `postgres`, password `chant`), in Docker, and prints both. `npx yodel emulator status` says whether they run, and `npx yodel emulator down` stops them. ## 3. Point the dev environment at it `chant.config.ts` names the `dev` environment's server and credentials by environment variable, and never holds a value. `dev` defaults to the emulator's address, so only the users and passwords are needed. The template gives each environment two users: a reader (the default, used by plans, pull request comments and drift checks) and a writer, picked with `YODEL_CREDENTIALS=writer` (only CI's apply job sets it, and `just apply` on your machine). yodel refuses an environment whose reader and writer are the same user, so give the emulator a read-only user for the reader, and keep its admin user, which can do everything, as the writer: #### ClickHouse ```sh curl -s http://127.0.0.1:8123 --data-binary "CREATE USER IF NOT EXISTS reader IDENTIFIED WITH no_password SETTINGS readonly = 1" curl -s http://127.0.0.1:8123 --data-binary "GRANT SELECT, SHOW ON *.* TO reader" export DEV_CLICKHOUSE_READER_USER=reader DEV_CLICKHOUSE_READER_PASSWORD= export DEV_CLICKHOUSE_WRITER_USER=default DEV_CLICKHOUSE_WRITER_PASSWORD= ``` The variables for every environment: | Variable | Meaning | |---|---| | `_CLICKHOUSE_URL` | the server's HTTP interface; `dev` defaults to `http://127.0.0.1:8123` | | `_CLICKHOUSE_READER_USER`, `_CLICKHOUSE_READER_PASSWORD` | the read-only user | | `_CLICKHOUSE_WRITER_USER`, `_CLICKHOUSE_WRITER_PASSWORD` | the user that applies, read when `YODEL_CREDENTIALS=writer` | #### Postgres The reader is a role of its own on the emulator, made once as `postgres`: ```sql CREATE ROLE reader LOGIN PASSWORD 'reader'; ALTER ROLE reader SET default_transaction_read_only = on; GRANT pg_read_all_data TO reader; ``` ```sh export DEV_POSTGRES_URL=postgres://127.0.0.1:5432/postgres export DEV_POSTGRES_READER_USER=reader DEV_POSTGRES_READER_PASSWORD=reader export DEV_POSTGRES_WRITER_USER=postgres DEV_POSTGRES_WRITER_PASSWORD=chant ``` The variables for every environment: | Variable | Meaning | |---|---| | `_POSTGRES_URL` | the server and the database, as `postgres://host:5432/database`; `dev` defaults to `postgres://127.0.0.1:5432/postgres` | | `_POSTGRES_READER_USER`, `_POSTGRES_READER_PASSWORD` | the read-only role | | `_POSTGRES_WRITER_USER`, `_POSTGRES_WRITER_PASSWORD` | the role that applies, read when `YODEL_CREDENTIALS=writer` | If a Postgres of your own already listens on `127.0.0.1:5432` (Homebrew's, for example), that address reaches it rather than the emulator. `just up` warns when it does. Stop that server, or reach the emulator through another address of this machine: `DEV_POSTGRES_URL=postgres://
:5432/postgres`, with the address from `ipconfig getifaddr en0` on macOS. `` is `DEV` or `PROD`. Without the users, `npx yodel plan dev` stops with `sql.profiles.dev.user names DEV_CLICKHOUSE_READER_USER, which is not set` (`DEV_POSTGRES_READER_USER` on Postgres). The reader cannot write, so `yodel apply` runs with `YODEL_CREDENTIALS=writer`, as `just apply` does. Run as the reader, `yodel apply dev` refuses before the approval gate (exit 4) and names the variable to set; the commands yodel prints to run again carry it too (`YODEL_CREDENTIALS=writer:dev npx yodel apply dev`). ## 4. Write the first migration The output on the rest of this page is from a run of the ClickHouse template; on Postgres the commands are the same. It was recorded before yodel printed `yodel approve` in its messages, so where an approval is asked for, the output shows chant's `chant approve` command, and the run approves with it, having no terminal to ask; you run `npx yodel approve dev`, which records the same approval. `yodel new` writes the next migration from the change in `src/` since the last one. The first one creates everything. `yodel lint` checks the migrations offline: ```console $ npx yodel new init Wrote migrations/20261010T2213-init/ (2 statements; the first migration) $ npx yodel lint 1 migration: 0 errors, 0 warnings, 0 silenced. ``` Commit it: ```sh git add -A && git commit -m "feat: the first migration" ``` ## 5. Plan, approve, apply `yodel plan` shows what `yodel apply` would run, and a digest of that plan: ```console $ npx yodel plan dev Plan for dev (clickhouse 26.8.15.10 at 127.0.0.1:8123, history yodeler.history (single (yodel.config.ts))): 0 applied, 1 pending 20261010T2213-init 0 statement SQLCH200 create events CREATE DATABASE events ENGINE = Atomic COMMENT 'Declared in src/schema.ts [chant managed-by=chant]' 1 statement SQLCH200 create events.events CREATE TABLE events.events ( id UInt64, kind LowCardinality(String), at DateTime ) ENGINE = MergeTree ORDER BY (kind, at) COMMENT 'One row per event [chant managed-by=chant]' The apply pipeline waits in wave 1 for an approval of jcs1-sha256:753d2acf4bf5873de6c3517ecc47c2b8c2dc5e285d9c54fb972ace5b7465be26 (gate always at 27fbeee6832b). Approve it for the pipeline with: chant approve yodel-apply yodel-apply-wave-1 --plan jcs1-sha256:753d2acf4bf5873de6c3517ecc47c2b8c2dc5e285d9c54fb972ace5b7465be26 Plan digest: jcs1-sha256:96585362be63615a9c5a29c059b2e211f409bcb3e3d5a20a1b226773dfea9aa3 Approve it with: chant approve migrate-dev approve-migrate-dev --plan jcs1-sha256:96585362be63615a9c5a29c059b2e211f409bcb3e3d5a20a1b226773dfea9aa3 or, at a terminal, with npx yodel approve dev, which shows the plan and asks you first. Not approved yet. Approving binds exactly this digest; if anything it covers changes before the apply runs, the gate asks again and nothing is applied. ``` The plan prints two approvals. The first, of the apply pipeline's wave, is for the CI pipeline's wave job after a merge, and you need it only once the project has a remote and CI. The second, of `migrate-dev`, is the one `yodel apply dev` waits for on your machine. `yodel apply` runs only a plan someone approved. The first run stops at the approval and prints the command that gives it (exit 3): ```console $ YODEL_CREDENTIALS=writer npx yodel apply dev Migrations in dev (clickhouse 26.8.15.10 at 127.0.0.1:8123, history yodeler.history, topology single (yodel.config.ts)) Applied: none Pending (1): 20261010T2213-init Plan digest: jcs1-sha256:96585362be63615a9c5a29c059b2e211f409bcb3e3d5a20a1b226773dfea9aa3 Running migrate-dev (/ops/migrate-dev.op.ts) [phase] Plan ✓ shellCmd(cmd=yodel apply dev --digest) 2.4s [phase] Approve • gate:approve-migrate-dev() skipped [phase] Apply • shellCmd(cmd=yodel apply dev --execute, env={"YODEL_APPROVED_PLAN":{"kind":"step-output-ref","step":"plan","path":"stdout"}}) skipped Op "migrate-dev" is gated on "approve-migrate-dev" after 2.9s plan : jcs1-sha256:96585362be63615a9c5a29c059b2e211f409bcb3e3d5a20a1b226773dfea9aa3 approve : chant approve migrate-dev approve-migrate-dev --plan jcs1-sha256:96585362be63615a9c5a29c059b2e211f409bcb3e3d5a20a1b226773dfea9aa3 expires : 2026-10-12T22:13:43.677Z note : recorded locally (no remote for chant/lifecycle) migrate-dev is waiting at gate "approve-migrate-dev" for approval of this plan (jcs1-sha256:96585362be63615a9c5a29c059b2e211f409bcb3e3d5a20a1b226773dfea9aa3). Approve it with: chant approve migrate-dev approve-migrate-dev --plan jcs1-sha256:96585362be63615a9c5a29c059b2e211f409bcb3e3d5a20a1b226773dfea9aa3 or, at a terminal, with npx yodel approve dev, which shows the plan and asks you first; then run YODEL_CREDENTIALS=writer:dev npx yodel apply dev again. [exit 3] ``` The note about a remote is expected: approvals live on a `chant/lifecycle` branch, and this repository has no remote to push it to. To approve, run `npx yodel approve dev`: it shows the plan and asks you to type `dev`. The command `yodel apply` prints adds the digest, `npx yodel approve dev --plan `, so it approves only the plan you were shown. The run below had no terminal to ask, so it recorded the same approval with chant's command. The approval holds for this exact plan only. Then run `yodel apply` again: ```console $ npx chant approve migrate-dev approve-migrate-dev --plan "$(npx yodel apply dev --digest)" Gate "approve-migrate-dev" on "migrate-dev" resolved by yodel at 2026-10-10T22:13:46.126Z (recorded locally (no remote for chant/lifecycle)) This approves the plan jcs1-sha256:96585362be63615a9c5a29c059b2e211f409bcb3e3d5a20a1b226773dfea9aa3, and only that plan. A run whose fresh plan differs refuses rather than applying it. This records the resolution as a fact; it does not itself re-run anything. Run `chant run migrate-dev` and it walks through gate "approve-migrate-dev". $ YODEL_CREDENTIALS=writer npx yodel apply dev Migrations in dev (clickhouse 26.8.15.10 at 127.0.0.1:8123, history yodeler.history, topology single (yodel.config.ts)) Applied: none Pending (1): 20261010T2213-init Plan digest: jcs1-sha256:96585362be63615a9c5a29c059b2e211f409bcb3e3d5a20a1b226773dfea9aa3 Running migrate-dev (/ops/migrate-dev.op.ts) lock: no KeeperMap on this server (KeeperMap is disabled because 'keeper_map_path_prefix' config is not defined. (BAD_ARGUMENTS) (version 26.8.15.10 (official build))); the lock is a file on this machine, so cross-runner locking needs Keeper (and keeper_map_path_prefix) approval: migrate-dev / approve-migrate-dev for jcs1-sha256:96585362be63615a9c5a29c059b2e211f409bcb3e3d5a20a1b226773dfea9aa3, by yodel at 2026-10-10T22:13:46.126Z 20261010T2213-init: applying statement 0: ok (SQLCH200 create, events) statement 1: ok (SQLCH200 create, events.events) 20261010T2213-init: applied [phase] Plan ✓ shellCmd(cmd=yodel apply dev --digest) 899ms [phase] Approve ✓ gate:approve-migrate-dev() 180ms [approved] yodel at 2026-10-10T22:13:46.126Z [phase] Apply ✓ shellCmd(cmd=yodel apply dev --execute, env={"YODEL_APPROVED_PLAN":"jcs1-sha256:96585362be63615a9c5a29c059b2e211f409bcb3e3d5a20a1b226773dfea9aa3"}) 2.0s Op "migrate-dev" completed in 3.2s Applied: 20261010T2213-init. $ npx yodel status dev Migrations in dev (clickhouse 26.8.15.10 at 127.0.0.1:8123, history yodeler.history, topology single (yodel.config.ts)) Applied (1): 20261010T2213-init 2026-10-10 22:13:49.991391 by yodel@example Pending: none ``` ## 6. A second migration Add a column to the table in `src/schema.ts`: ```ts at DateTime, country LowCardinality(String) DEFAULT '' ``` Then write the migration, check it and commit it: ```console $ npx yodel new add-country Wrote migrations/20261010T2213-add-country/ (1 statement; follows 20261010T2213-init) $ npx yodel lint 2 migrations: 0 errors, 0 warnings, 0 silenced. ``` ```sh git add -A && git commit -m "feat: events.country" ``` The plan has the one new statement. The approval you gave for the first migration does not hold for this plan, and the plan says why: ```console $ npx yodel plan dev Plan for dev (clickhouse 26.8.15.10 at 127.0.0.1:8123, history yodeler.history (single (yodel.config.ts))): 1 applied, 1 pending 20261010T2213-add-country 0 statement SQLCH201 metadata events.events ALTER TABLE `events`.`events` ADD COLUMN country LowCardinality(String) DEFAULT '' AFTER `at` The apply pipeline waits in wave 1 for an approval of jcs1-sha256:2cef01bec2fd0e3182fcb24fe645ec39f48697c3573f26338e064c459f656203 (gate always at b3e7a48d268e). Approve it for the pipeline with: chant approve yodel-apply yodel-apply-wave-1 --plan jcs1-sha256:2cef01bec2fd0e3182fcb24fe645ec39f48697c3573f26338e064c459f656203 Plan digest: jcs1-sha256:3df448990a3d23ff2213501d47ac943e9acc6267e1c0fa134e012ac6ab97af8a Approve it with: chant approve migrate-dev approve-migrate-dev --plan jcs1-sha256:3df448990a3d23ff2213501d47ac943e9acc6267e1c0fa134e012ac6ab97af8a or, at a terminal, with npx yodel approve dev, which shows the plan and asks you first. The newest approval of approve-migrate-dev, by yodel at 2026-10-10T22:13:46.126Z, is for jcs1-sha256:96585362be63615a9c5a29c059b2e211f409bcb3e3d5a20a1b226773dfea9aa3, not this plan; it does not hold. What moved (as far as yodel can tell): history changed: applied since: 20261010T2213-init (by yodel@example at 2026-10-10 22:13:49.991391); pending migrations changed: committed to since approval: 20261010T2213-add-country (1ee6e61 at 2026-10-10T16:13:53-06:00); live schema changed: as expected after the history change above: applying a migration changes the live schema, and yodel cannot rebuild the live schema as it was at approval, so it cannot confirm that nothing else in it changed. ``` Approve and apply as before: ```console $ npx chant approve migrate-dev approve-migrate-dev --plan "$(npx yodel apply dev --digest)" Gate "approve-migrate-dev" on "migrate-dev" resolved by yodel at 2026-10-10T22:13:57.553Z (recorded locally (no remote for chant/lifecycle)) This approves the plan jcs1-sha256:3df448990a3d23ff2213501d47ac943e9acc6267e1c0fa134e012ac6ab97af8a, and only that plan. A run whose fresh plan differs refuses rather than applying it. This records the resolution as a fact; it does not itself re-run anything. Run `chant run migrate-dev` and it walks through gate "approve-migrate-dev". $ YODEL_CREDENTIALS=writer npx yodel apply dev Migrations in dev (clickhouse 26.8.15.10 at 127.0.0.1:8123, history yodeler.history, topology single (yodel.config.ts)) Applied (1): 20261010T2213-init 2026-10-10 22:13:49.991391 by yodel@example Pending (1): 20261010T2213-add-country Plan digest: jcs1-sha256:3df448990a3d23ff2213501d47ac943e9acc6267e1c0fa134e012ac6ab97af8a Running migrate-dev (/ops/migrate-dev.op.ts) lock: no KeeperMap on this server (KeeperMap is disabled because 'keeper_map_path_prefix' config is not defined. (BAD_ARGUMENTS) (version 26.8.15.10 (official build))); the lock is a file on this machine, so cross-runner locking needs Keeper (and keeper_map_path_prefix) approval: migrate-dev / approve-migrate-dev for jcs1-sha256:3df448990a3d23ff2213501d47ac943e9acc6267e1c0fa134e012ac6ab97af8a, by yodel at 2026-10-10T22:13:57.553Z 20261010T2213-add-country: applying statement 0: ok (SQLCH201 metadata, events.events) 20261010T2213-add-country: applied [phase] Plan ✓ shellCmd(cmd=yodel apply dev --digest) 947ms [phase] Approve ✓ gate:approve-migrate-dev() 216ms [approved] yodel at 2026-10-10T22:13:57.553Z [phase] Apply ✓ shellCmd(cmd=yodel apply dev --execute, env={"YODEL_APPROVED_PLAN":"jcs1-sha256:3df448990a3d23ff2213501d47ac943e9acc6267e1c0fa134e012ac6ab97af8a"}) 2.0s Op "migrate-dev" completed in 3.3s Applied: 20261010T2213-add-country. ``` ## 7. Drift `yodel drift` compares what `src/` declares with the server, and reports anything changed by hand. Change a default on the server, as someone might by hand: ```sql ALTER TABLE events.events MODIFY COLUMN country LowCardinality(String) DEFAULT 'US' ``` `yodel drift dev` reports it, and exits 2: ```console $ npx yodel drift dev Drift in dev (clickhouse 26.8.15.10 at 127.0.0.1:8123) Compared with the schema 20261010T2213-add-country records, the newest migration the history records as applied. Changed out of band: events (ClickHouse::Table) column country default.expr: declared '', live 'US' [exit 2] ``` To put it right, change it back on the server, or declare the new default in `src/schema.ts` and write a migration for it. ## The same steps with just Each template has a `justfile` for these steps. [just](https://github.com/casey/just) is optional: every target runs the command beside it, and the commands work without it. | just | Runs | |---|---| | `just up` | `npx yodel emulator up`, then `npx tsx scripts/local.ts`, which checks that `dev` reaches the emulator, makes the read-only user `reader` there (the statements in step 3), and prints the reader and writer lines for a `.env` file | | `just down` | `npx yodel emulator down` | | `just new ` | `npx yodel new ` | | `just lint` | `npx yodel lint` | | `just replay` | `npm run replay`: every migration replayed into a throwaway server and compared with its recorded schema | | `just plan [env]` | `npx yodel plan ` (`dev` when left out) | | `just approve [env]` | prints the command that approves the environment's plan, for you to run; it never runs it | | `just apply [env]` | `YODEL_CREDENTIALS=writer npx yodel apply ` | | `just status [env]` | `npx yodel status ` | | `just drift [env]` | `npx yodel drift ` | | `just ci` | `npm run ci`, which renders the CI pipelines again | The justfile reads a `.env` file in the project when there is one, so the variables from step 3 can go there in place of `export`; `.gitignore` keeps it out of git. Only `just` reads it: the plain `npx yodel ...` commands above see only the shell's environment, so to run them with the variables in `.env`, load it first with `set -a; . ./.env; set +a`. ## Next - Push the project and set up CI: [Setting up each forge](/sql-yodeler/forges/) has the secrets, tokens and runner each forge needs. From then on each change is a pull request: edit `src/schema.ts`, `npx yodel new `, commit both. The pull request gets a plan comment, and a merge applies what was approved. - [The two workflows](/sql-yodeler/workflows/), [Migrations](/sql-yodeler/migrations/) and [Approval](/sql-yodeler/approval/) explain what you just ran. - [Drift](/sql-yodeler/drift/) has the scheduled drift check the pipelines run. - To take a database you already have into migrations, see [Starting a project](/sql-yodeler/adoption/#adopting-an-existing-database-yodel-init-from). ## Proven by The scenario claims below run what this page describes against a real server, once plain (it passes) and once with the behaviour broken (the claim catches it). [Claims status](/sql-yodeler/claims/) lists every claim. | Claim | What it says | Plain, broken | Last run | |---|---|---|---| | `new` | yodel new writes the next migration offline, with no dev database, and it applies; on Postgres, functions, procedures and triggers too, which lint checks | ClickHouse: pass, caught; Postgres: pass, caught | `c6f58a4`, 2026-10-10 | | `lint` | yodel lint --replay replays the migrations into a fresh database and each gives the schema it recorded; offline, yodel lint fails a migrations directory with a fork (exit 3), naming both migrations; a checkpoint replays alone to its recorded schema, and a fresh environment starts from it; yodel revert undoes the newest migration behind the gate, with a hand-written step for its data step, back to its parent's recorded schema, and refuses a checkpoint or a migration before one; yodel test runs a project's tests on databases replayed from the migrations, a seed meeting the backfill after it, fails a case whose assertion does not hold, and refuses an environment with a history; on Postgres, a unique index over rows with duplicates is refused before the gate by its generated pre-check (exit 4, naming the statement and the count), and yodel lint flags it as data-dependent | ClickHouse: pass, caught; Postgres: pass, caught | `c6f58a4`, 2026-10-10 | | `approval` | a pending migration applies only after chant approve of its plan digest, and the history records that digest; a failing pre-migration check refuses it, and so does a policy rule, read at the base commit, unless an override is recorded for that digest, for a yodel revert as for an apply; the audit log derived from the history and the ledger accounts for every approval and apply, and names one removed; a reader and a writer that are one user are refused, and the reader, its password a minted token, cannot write; an approval is refused once the migration changed after it, and nothing is applied | ClickHouse: pass, caught; Postgres: pass, caught | `c6f58a4`, 2026-10-10 | | `drift` | yodel drift reports a declared object changed out of band, naming the property, and one dropped; on the versioned path it compares with the newest applied migration's recorded schema, so a pending migration is not drift | ClickHouse: pass, caught; Postgres: pass, caught | `c6f58a4`, 2026-10-10 | --- # Starting a project Source: https://intentius.io/sql-yodeler/adoption/ ## Optional: hand this page to your coding agent ```text Take the existing database of the environment I name into versioned migrations with SQL Yodeler, following https://intentius.io/sql-yodeler/adoption/. Make the project from the starter template with the existing database or schema name first, and ask me before running `YODEL_CREDENTIALS=writer npx yodel init --from --force`: it writes one row to that environment's history. Open a pull request with src/, the baseline migration and the config. Never run `yodel apply` against a shared environment, never run `chant approve`, `yodel approve` or `yodel override`, never edit the `chant/lifecycle` branch or `.chant/allowed_signers`, never merge; approvals and applies belong to people. ``` There are two ways in: a new project from a starter template, whose first migration creates everything, or `yodel init --from `, which adopts a database that already exists. ## A new project from a template `templates/clickhouse` and `templates/postgres` in this repository are starter projects, and `yodel create` makes a project from one. [Your first migration](/sql-yodeler/getting-started/) takes a project made this way to applied migrations on a local database, step by step. Each gives `chant.config.ts` with a `dev` and a `prod` profile, `yodel.config.ts`, a migrations Op and a drift watch per environment, and CI pipelines for GitHub Actions, GitLab CI and Forgejo Actions. `yodel create --help` lists the options. Before the project exists, run yodel from npm: ```sh npx @intentius/sql-yodeler@latest create my-schema --clickhouse --database events npx @intentius/sql-yodeler@latest create my-schema --postgres --schema app ``` The template is the one released with that yodel (tag `v` of this repository), so the project matches the yodel that made it. `--name` sets the package name, the directory's name by default. The transcript below predates `yodel create`: it made the project from a clone's template, shown as ``, with chant's CLI, which `yodel create` runs underneath. The project is the same. ```console $ npx chant init --from /templates/clickhouse --param name=events-schema --param database=events my-schema Created: .forgejo/workflows/watch-dev.yml .forgejo/workflows/watch-prod.yml .forgejo/workflows/yodel-apply.yml .forgejo/workflows/yodel-pr.yml .github/workflows/watch-dev.yml .github/workflows/watch-prod.yml .github/workflows/yodel-apply.yml .github/workflows/yodel-pr.yml .gitignore .gitlab-ci.yml .gitlab/yodel-apply.gitlab-ci.yml .gitlab/yodel-watch.gitlab-ci.yml chant.config.ts justfile migrations/.gitkeep ops/migrate-dev.op.ts ops/migrate-prod.op.ts ops/watch-dev.op.ts ops/watch-prod.op.ts package.json README.md scripts/local.ts src/schema.ts tsconfig.json yodel-waves.json yodel.config.ts .chant/workspace.lock.json Lineage: dir:/templates/clickhouse (a directory, recorded by digest only), recorded in .chant/workspace.lock.json Parameters: database="events", history_database="yodeler", name="events-schema" ``` The template's `package.json` takes `@intentius/sql-yodeler` from npm (`^0.5.2`), so `npm install` in `my-schema` needs no token, and neither do the generated pipelines' `npm ci`; [Installing](/sql-yodeler/configuration/#installing) has the other routes. The run quoted here linked the clone's `node_modules` into `my-schema` in place of `npm install`. The first migration creates what `src/schema.ts` declares. `yodel new` needs no server: ```console $ npx yodel new init Wrote migrations/20261010T1725-init/ (2 statements; the first migration) $ npx yodel lint 1 migration: 0 errors, 0 warnings, 0 silenced. ``` Make the directory a git repository and commit (`yodel apply` finds the migrations Op through git, and chant keeps approvals on the `chant/lifecycle` branch). Then create the database users the template's README lists, add the forge's secrets as [Setting up each forge](/sql-yodeler/forges/) says, and push. From there each change is a pull request: edit `src/schema.ts`, run `npx yodel new `, commit both. [The two workflows](/sql-yodeler/workflows/) and [Approval](/sql-yodeler/approval/) go on from here. ## Adopting an existing database: yodel init --from `yodel init --from ` takes a database that already exists into versioned migrations in one command: 1. It reads the live database the profile names (chant's import, as `chant import --from ` runs it) and writes the declarations to the source directory (`sourceDir`, else `src/`). 2. It builds them and plans them against the same database. The plan must show no change; if it does not, everything it wrote is taken back. 3. It writes the baseline migration, `migrations/-baseline/`: the first migration, whose statements create everything. 4. It records the baseline applied in the environment's history, without running any of its statements. On Postgres, when the environment manages access (`environments..access` in `yodel.config.ts`, or `sql.profiles..access`), step 1 also reads each table's row-level security, the policies and the privileges, and declares them; roles stay the environment's and are named as text ([Access control](/sql-yodeler/access/#adopting-a-database)). On ClickHouse it reads the row policies on the tables, the users and roles that hold privileges there, and every grant each of them holds, and writes them to `src/access.ts`; a user's password stays the environment's ([Access control](/sql-yodeler/access/#adopting)). Dictionaries are adopted with the tables whether access is managed or not, and so are the SQL functions an adopted view or column default calls and those `sql.profiles..importFunctions` names ([Workflows](/sql-yodeler/workflows/)). The live database is only read. The one write is the history row (and the history's database or schema and table, when they are not there yet). What it needs first: - A project: a `chant.config.ts` with `lexicons: ["sql"]`, `sql.dialect` and `sql.profiles.`, with the project's databases (ClickHouse, `databases`) or schemas (Postgres, `schemas`) listed and the history's database or schema not among them, and a `package.json` from which chant and the `sql` lexicon resolve. `init` does not write `chant.config.ts`. The usual way to get one is a [starter template](#a-new-project-from-a-template) made with the existing database's name (`yodel create --clickhouse --database `, or `--postgres --schema `); its `src/schema.ts` is a placeholder, so run `init` with `--force` to write over it. - The writer's credentials. `init` writes the history row, and creates the history's database or schema and table when they are not there. The templates read the reader's credentials, which their READMEs create read-only, unless `YODEL_CREDENTIALS=writer`, so with a template it is `YODEL_CREDENTIALS=writer npx yodel init --from --force`. - No subdirectory in `migrations/`. Move another tool's migration files out of it first. It refuses (exit 4, nothing written) when the source directory already declares a schema (unless `--force`), the project already has migrations, the history already records migrations, or the environment holds nothing. `init` adopts every object in the profile's databases or schemas, except the history tables of other migration tools (golang-migrate's and dbmate's `schema_migrations`, goose's `goose_db_version`, Flyway's `flyway_schema_history`, and others), which it leaves out with a warning. There is no exclude list: a table the project should not own belongs in a database or schema the profile does not list. [Coming from another migration tool](/sql-yodeler/from-other-tools/) covers adopting a database another tool manages: the old tool's files and table, more than one environment, cutting CI over, and rollback. Both examples adopt a database this way. The ClickHouse example's run, where the database was first made on the declarative path, so `src/schema.ts` was there and `--force` writes over it: ```console $ npx yodel init --from dev --force read 2 object(s) from dev wrote src/schema.ts planned against dev: no change wrote migrations/20261010T1722-baseline (2 statements) lock: no KeeperMap on this server (KeeperMap is disabled because 'keeper_map_path_prefix' config is not defined. (BAD_ARGUMENTS) (version 26.8.15.10 (official build))); the lock is a file on this machine, so cross-runner locking needs Keeper (and keeper_map_path_prefix) recorded 20261010T1722-baseline applied in yodeler.history as a baseline; none of its statements was run Adopted dev (clickhouse): 2 objects Database shop Table shop.events Declarations: src/schema.ts Baseline: migrations/20261010T1722-baseline/ (2 statements, sha256:571b633c0d6c4e461d76285328cb71600b3c33c1a429f90db1a5f9e174228deb) Recorded: applied in yodeler.history as a baseline; none of its statements was run yodel plan dev shows no change, and yodel status dev lists the baseline applied. ``` The declarations it wrote: ```ts import { database, table } from "@intentius/chant-lexicon-sql/clickhouse"; export const shopDb = database` CREATE DATABASE shop ENGINE = Atomic COMMENT 'The SQL Yodeler ClickHouse example'`; export const events = table` CREATE TABLE ${shopDb}.events ( id UInt64, kind LowCardinality(String), at DateTime ) ENGINE = MergeTree PARTITION BY toYYYYMM(at) ORDER BY (kind, at)`; ``` and the history: ```console $ npx yodel status dev Migrations in dev (clickhouse 26.8.15.10 at 127.0.0.1:8123, history yodeler.history, topology single (default)) Applied (1): 20261010T1722-baseline 2026-10-10 17:22:07.370595 by yodel@example Pending: none ``` The baseline's statements are the ones that would create the database from nothing, so a new environment (a fresh dev database, the replay check's throwaway server) gets everything by applying it. On the adopted environment it is never run. Objects made by hand carry no chant ownership marker in their comments, and `init` does not add one: they stay unmarked until a declarative apply stamps them. The plan compares declared definitions without the marker, so they plan clean either way. After `init`, `yodel apply ` needs a migrations Op for the environment (an Op with a Plan step running `yodel apply --digest`, a gate bound to its output, and an Apply step running `yodel apply --execute`); `yodel apply` prints the declaration to add when there is none. See [Approval](/sql-yodeler/approval/). The Postgres example does the same on Postgres. ## Another environment that already holds the schema: yodel init --baseline `init --from` adopts one environment. Another environment that holds the same schema with its data (staging, prod) cannot run the baseline, whose first `CREATE` would fail there. `yodel init --baseline ` records the baseline applied in it instead, without running it: ```sh YODEL_CREDENTIALS=writer npx yodel init --baseline prod ``` 1. It plans the schema the baseline (the project's first migration) records against ``'s live database, from `migration.json` alone. The plan must be empty: every object the baseline declares is there as it declares it, and the profile's databases or schemas hold nothing else. Each difference is named, with what the environment has and what the baseline declares, and the command refuses (exit 4), recording nothing: ```text yodel init: refused: prod's live schema is not the schema 20261010T1745-baseline records, so the baseline cannot be recorded applied there. 1 difference: customers (app.customers) columns.extra: extra text in the environment, (none) in the baseline (SQLPG204) Make prod hold the baseline's schema (or empty it, and let yodel apply run the baseline), then run yodel init --baseline prod again. Nothing was recorded. ``` 2. When they match, it runs ``'s migrations Op, as `yodel apply` does. The Plan step compares again and prints the record's digest (the baseline, the empty history and the live schema), the gate binds the approval to that digest, and the run stops there (exit 3) with the command that approves it, `yodel approve --plan `. Approve, and run `yodel init --baseline ` again. 3. The Apply step takes the lock, checks that the history still records nothing, compares again, refuses if the digest moved since the approval, and appends the baseline's history row: a succeeded migration row whose `note` starts `baseline:`, with the approved digest. None of the baseline's statements is sent. From there `yodel status ` lists the baseline applied, and `yodel plan ` and `yodel apply ` take the migrations after it, as in every other environment. It refuses (exit 4) a project with no migrations and an environment whose history records anything. ## Proven by The scenario claims below run what this page describes against a real server, once plain (it passes) and once with the behaviour broken (the claim catches it). [Claims status](/sql-yodeler/claims/) lists every claim. | Claim | What it says | Plain, broken | Last run | |---|---|---|---| | `adopt` | yodel init --from adopts a live database without touching it, and yodel plan then shows no change; on Postgres its policies, row-level security and grants too, on ClickHouse its dictionaries, functions, roles, users, row policies and grants; yodel init --baseline records the baseline, behind the gate, in a second environment that holds the same schema, and refuses one that differs | ClickHouse: pass, caught; Postgres: pass, caught | `9329873`, 2026-10-10 | | `new` | yodel new writes the next migration offline, with no dev database, and it applies; on Postgres, functions, procedures and triggers too, which lint checks | ClickHouse: pass, caught; Postgres: pass, caught | `c6f58a4`, 2026-10-10 | --- # Coming from another migration tool Source: https://intentius.io/sql-yodeler/from-other-tools/ ## Optional: hand this page to your coding agent ```text Move this repository's database migrations from the tool it uses now to SQL Yodeler, following https://intentius.io/sql-yodeler/from-other-tools/. Make the project from the starter template, move the old migration files out of migrations/, and ask me before running `YODEL_CREDENTIALS=writer npx yodel init --from --force` against the environment I name: it writes one row to that environment's history. Leave the old tool's history table in the database. Open a pull request with src/, the baseline migration, the config and the pipelines, with the old tool's migration jobs removed and listed in its description. Never run `yodel apply` against a shared environment, never run `chant approve`, `yodel approve` or `yodel override`, never edit the `chant/lifecycle` branch or `.chant/allowed_signers`, never merge; approvals and applies belong to people. ``` This page is for a database that golang-migrate, goose, Flyway or dbmate manages today. It maps their concepts onto SQL Yodeler's, takes the database into versioned migrations with its current schema as the starting point, and says how rollback works. ## The concepts, side by side The main difference in day-to-day work: you do not write the migration's SQL. You change the declared schema in `src/` (chant's `sql` lexicon), and `yodel new ` writes the migration from the difference between that and the schema the previous migration recorded. You can still edit `migration.sql` by hand before it is applied anywhere; `yodel lint --update-checksum ` then records the new checksum. | In golang-migrate, goose, Flyway, dbmate | In SQL Yodeler | |---|---| | An up file: `1_add_note.up.sql`, `V2__add_note.sql`, a `-- +goose Up` or `-- migrate:up` section | A directory, `migrations/-/`, with `migration.sql` (the statements) and `migration.json` (its parent, its checksum, its steps and the schema as it stands after it). `yodel new ` writes it. See [Migrations](/sql-yodeler/migrations/#the-directory). | | A down file, `-- +goose Down`, `-- migrate:down`, `flyway undo` | No down files. `yodel revert ` plans the reverse from the recorded schemas when you need it ([below](#rolling-back)). | | The order: the version number or timestamp in the file name | Each migration names its parent; the chain of parents is the order. The timestamp in the id is for people and never decides the order. | | `schema_migrations` (version, dirty), `goose_db_version`, `flyway_schema_history` | `.history`, in a database (ClickHouse) or schema (Postgres) of its own, `yodeler` by default. It is append-only: one row per event (a migration or statement started, succeeded, failed), with who ran it and under which approval. See [The history table](/sql-yodeler/migrations/#the-history-table). | | `migrate version`, `goose status`, `flyway info`, `dbmate status` | `yodel status `: applied, pending, failed part way, out of order, checksum mismatches. | | golang-migrate's dirty flag and `migrate force`; a failed row in `flyway_schema_history` | A migration that failed part way. The history records each statement, so the next `yodel apply` resumes at the statement that failed and never sends again one that succeeded. There is nothing to force. See [Resuming](/sql-yodeler/migrations/#resuming-a-migration-that-failed-part-way). | | `flyway validate` | `yodel status` and `yodel apply` check every applied migration's files against the checksum recorded when it ran, and refuse on a mismatch; `yodel lint` checks each migration's own `checksum` field. | | `flyway repair` | `yodel repair --reason ""` records a new checksum for an applied migration whose files were edited on purpose, with the reason. Nothing runs on the server. It does not clear failures: a failed migration resumes instead. | | `baseline`, `baselineOnMigrate` | `yodel init --from `: reads the live database into declarations, writes a baseline migration whose statements create all of it, and records it applied without running it. See [Adopting it](#adopting-the-database). | | `outOfOrder=true`, goose's `--allow-missing` | `yodel apply --allow-out-of-order`, for one run. Without it, a pending migration the chain puts before an applied one is refused (exit 4), never skipped. See [Out-of-order migrations](/sql-yodeler/migrations/#out-of-order-migrations). | | Flyway's repeatable migrations (`R__`) for views and functions | Views, functions, procedures and triggers are declared in `src/` like tables. A changed body becomes a `CREATE OR REPLACE` in the next migration ([Postgres objects](/sql-yodeler/workflows/#postgres-objects)). | | goose's Go migrations, Flyway's Java migrations: code that moves data | Steps inside a migration: a backfill (`yodel new --backfill `), a ClickHouse rebuild, a Postgres expand-and-contract change. They resume from their receipts. See [Data migrations](/sql-yodeler/steps/). | | Flyway callbacks (`beforeMigrate`, `afterMigrate`) | `environments..steps` in `yodel.config.ts`: read-only checks and commands before and after each apply ([Steps around apply](/sql-yodeler/approval/#steps-around-apply-checks-before-and-after)). | | dbmate's `schema.sql` dump | Each `migration.json` records the schema after it, and `src/` declares the current one. `yodel docs` writes an HTML reference and an ERD ([Generated schema reference](/sql-yodeler/schema-docs/)). | | Squashing old migrations into one | `yodel checkpoint `: new environments start from it instead of replaying the whole chain ([Checkpoints](/sql-yodeler/migrations/#checkpoints-yodel-checkpoint)). | | `migrate up` / `flyway migrate` in a deploy job | `yodel apply ` in the pipelines `yodel ci` renders: the plan is posted on the pull request, and the apply after a merge waits for an approval bound to a digest of that plan ([Approval](/sql-yodeler/approval/)). | | Tests that run the migrations on a fresh database | `yodel test`: cases in `tests/*.test.ts`, run on a database built from the migrations ([Lint](/sql-yodeler/lint/#tests-on-the-replayed-database-yodel-test)). | ## Adopting the database Adoption keeps the database as it is. The migrations the old tool ran are not imported one by one: the schema they produced becomes the baseline, the first migration of the new chain, recorded as applied without running. ### Before you start 1. Stop the old tool's migration jobs for the environment, and apply whatever the old tool still has pending, so the database holds the schema you mean to keep. A migration the old tool runs after the baseline is recorded changes the database behind yodel's back: `yodel drift` reports a declared object it changed or dropped, and a table it created stays undeclared. 2. Decide which environment to adopt with `yodel init --from`, usually production. The others that hold the same schema take the baseline afterwards with `yodel init --baseline` (see [More than one environment](#more-than-one-environment)). 3. Have the writer's credentials for it. `init` reads the database and then writes one row to the history (creating the history's database or schema and table first, when they are not there), so the user needs to create those: on ClickHouse, `CREATE DATABASE` (or an existing history database it can create tables in); on Postgres, `CREATE` on the database. A read-only user fails at that write. The plan check before it only reads. ### The project `yodel init` needs a project around it: `chant.config.ts` with a profile for the environment, `yodel.config.ts`, a migrations Op, and a `package.json` from which chant and the `sql` lexicon resolve. It does not write these. The quickest way to get them is a starter template, made with `yodel create` and the database (ClickHouse) or schema (Postgres) you already have: ```sh npx @intentius/sql-yodeler@latest create events-schema --clickhouse --database events npx @intentius/sql-yodeler@latest create app-schema --postgres --schema app cd app-schema && npm install ``` The history goes in its own database or schema (`--history ` on `yodel create`, `yodeler` by default); it must not be one of the project's. When the project owns more than one database or schema, list each in `databases` (ClickHouse) or `schemas` (Postgres) of every profile in `chant.config.ts`. [Starting a project](/sql-yodeler/adoption/#a-new-project-from-a-template) has the rest of what the template writes. ### Moving the old tool's files yodel's migrations live in `migrations/` at the project root, and the name is fixed. Move the old tool's files out of it before you run `init` (for example to `legacy-migrations/`, or delete them and let git keep them). `init` refuses while `migrations/` holds any subdirectory, and yodel never reads the old files: flat `.sql` files left there are ignored, which only confuses whoever reads the directory later. ### Running init Point the environment at the database (in the templates, `_CLICKHOUSE_URL` or `_POSTGRES_URL`, and the writer's user and password variables), then: ```sh YODEL_CREDENTIALS=writer npx yodel init --from prod --force ``` - `YODEL_CREDENTIALS=writer` makes the template's `chant.config.ts` read the writer's variables. Without it the template reads the reader's, which its README creates read-only. - `--force` is needed because the template's `src/schema.ts` is a placeholder schema, and `init` refuses to write over declarations without it: `refused: src already declares a schema (schema.ts) ... Pass --force to overwrite`. The import writes `src/schema.ts`; any other file in `src/` is kept and built with it, so remove files that declare objects the database does not have. What `init` does, step by step, and what it refuses, is in [Adopting an existing database](/sql-yodeler/adoption/#adopting-an-existing-database-yodel-init-from). Afterwards `npx yodel plan prod` shows no change and `npx yodel status prod` lists the baseline applied. Commit `src/`, the baseline and the project. ### The old tool's history table `init` leaves the old tool's history table out of the declared schema, with a warning such as `app.schema_migrations is kept by a migration runner (Rails, golang-migrate, dbmate); left out, since declaring it would have chant change what that tool owns`. It does this on both dialects for the tables chant knows by name: `schema_migrations`, `goose_db_version`, `flyway_schema_history`, and those of Prisma, Rails, Django, Alembic, drizzle-kit, Knex, Sequelize and node-pg-migrate. (Flyway keeps its table under a configurable name; one under another name is adopted like any other table.) From then on yodel never reads it or writes to it: migrations are written from the declared schema, and drift reports only declared objects. Keep it while you might go back to the old tool, and drop it by hand once nothing runs the old tool. Its rows changing does not move a plan's digest; creating or dropping the table in one of the project's databases or schemas does, so drop it between applies, not while an approved plan is waiting. ### Tables the project should not own There is no exclude list. The scope is the databases (ClickHouse) or schemas (Postgres) the profile lists: `init` adopts every object in them except the history tables above, and nothing outside them. To keep a table out of the project (a table another service owns, a scratch table), keep it in a database or schema the profile does not list. Deleting a table's declaration after `init` does not undeclare it: the baseline recorded it, so the next `yodel new` writes a `DROP TABLE`, which `yodel lint` flags as destructive. An undeclared object in a listed database or schema (one created after `init`) is left alone: migrations are written from the declarations, and drift does not report it. Creating or dropping one does move the plan digest, since the digest covers the catalog of those databases or schemas. ### More than one environment Once `migrations/` holds the baseline, `init --from` refuses in that project, so it records the baseline in one environment. An environment that does not hold the schema yet, or one you can empty first (a dev or CI database), starts with an empty history, and `yodel apply` runs the baseline there, which creates everything. An environment that already holds the schema and its data takes it with `yodel init --baseline `: it compares that environment's live schema with the schema the baseline records, refuses and names each difference when they differ, and otherwise records the baseline applied there, without running it, behind the environment's gate ([Another environment that already holds the schema](/sql-yodeler/adoption/#another-environment-that-already-holds-the-schema-yodel-init-baseline)). Stop the old tool in that environment first too, and bring it to the same schema as the adopted one. ### Cutting CI over The template renders pipelines for GitHub Actions, GitLab CI and Forgejo Actions (`npm run ci`, which is `yodel ci`): lint and a plan comment on each pull request with the reader's credentials, and on a merge, an apply per environment with the writer's, behind its approval. Create the users the template's README lists and the secrets [Setting up each forge](/sql-yodeler/forges/) places, remove the old tool's migration jobs in the same pull request that adds these, and from then on make each change by editing `src/` and running `yodel new `. [The two workflows](/sql-yodeler/workflows/#the-pipelines-yodel-ci) has the details. ## Rolling back There are no down migrations to write or keep in step with the up ones. Each migration records the schema after it, and its parent records the schema before it, so yodel can plan the reverse when you ask for it. ### Reverting the newest migration ```sh npx yodel revert prod --dry-run # the reverse statements, each with its rule and class, and the digest YODEL_CREDENTIALS=writer npx yodel revert prod ``` `yodel revert` undoes the newest migration applied in the environment, and only that one, since any later migration was written against it. It goes through the migrations Op like an apply: it stops at the gate with the approve command (exit 3), and once approved, the next `yodel revert` runs it under the lock. The policy in `yodel.config.ts` judges its statements, so a rule that refuses drops refuses a revert that drops what the migration added unless an override is recorded. The history gets a row for each reverse statement and a `reverted` row; the migration is then pending again, and `yodel status` marks it reverted. Delete it, or rewrite it with `yodel new --replace`, before the next `yodel apply`, which would otherwise apply it again. A revert carries the schema back, not the data. A dropped column comes back empty, and a revert that drops a column the migration added loses what was written to it since. Reverting the baseline plans the drop of everything it created. ### When a hand-written step is needed - The migration has a data step (a backfill). The revert is refused (exit 2) until `--step ` names SQL that undoes the data step; it runs before the reverse statements and is part of the digest. - The reverse would need an Op or a manual step (a ClickHouse sort key changed back, a Postgres column renamed back). The revert is refused (exit 2). Write the change back as a new migration: edit `src/`, then `yodel new `. - The migration is not the newest applied one, or other environments have moved on from it. Roll forward the same way: a new migration that undoes it. ### A migration that failed part way It is not rolled back. Statements that succeeded stay, and the next `yodel apply` resumes at the one that failed. Fix the cause on the server (the row a constraint rejects, a lock someone holds), or edit that statement in `migration.sql` and record the files' new checksum with `yodel lint --update-checksum `; then approve and apply again. `yodel repair` is for a migration that applied completely and whose files were edited afterwards; it refuses one that failed part way. On Postgres, each statement that can run in a transaction runs in its own, so a failed statement leaves nothing half done. ## Proven by The scenario claims below run what this page describes against a real server, once plain (it passes) and once with the behaviour broken (the claim catches it). [Claims status](/sql-yodeler/claims/) lists every claim. | Claim | What it says | Plain, broken | Last run | |---|---|---|---| | `adopt` | yodel init --from adopts a live database without touching it, and yodel plan then shows no change; on Postgres its policies, row-level security and grants too, on ClickHouse its dictionaries, functions, roles, users, row policies and grants; yodel init --baseline records the baseline, behind the gate, in a second environment that holds the same schema, and refuses one that differs | ClickHouse: pass, caught; Postgres: pass, caught | `9329873`, 2026-10-10 | | `resume` | an interrupted apply resumes where it stopped: a backfill step written as yodel's form (table, key, batch size, SQL) from its receipts, running each batch once, and a failed statement at that statement, never resending one that ran | ClickHouse: pass, caught; Postgres: pass, caught | `c6f58a4`, 2026-10-10 | | `out-of-order` | a migration merged late, before one already applied, is refused unless --allow-out-of-order, and never skipped | ClickHouse: pass, caught; Postgres: pass, caught | `c6f58a4`, 2026-10-10 | --- # Lint and plan on pull requests only Source: https://intentius.io/sql-yodeler/lint-and-plan/ ## Optional: hand this page to your coding agent ```text Set up SQL Yodeler's pull request checks without its apply pipeline, following https://intentius.io/sql-yodeler/lint-and-plan/: set `ci: { apply: false }` in yodel.config.ts, run `npm run ci`, delete the apply files it no longer writes, and check that `npx yodel ci --check` passes. List the readers' secrets the forge needs in the pull request description, and add no writer's secret. Open a pull request with the result. Never run `yodel apply` against a shared environment, never run `chant approve`, `yodel approve` or `yodel override`, never edit the `chant/lifecycle` branch or `.chant/allowed_signers`, never merge; approvals and applies belong to people. ``` This page is for a team that wants SQL Yodeler to review migrations in pull requests before it lets SQL Yodeler apply anything. Every pull request gets `yodel lint`, the replay check and a plan comment per environment, and the migrations are applied some other way for now: from a terminal with `yodel apply`, or with the process the team already has. CI holds read-only credentials only, and no job in it can write to a database. ## Setting it up 1. Make the project. [Starting a project](/sql-yodeler/adoption/) covers both ways in: a new project from a starter template, or `yodel init --from ` for a database that already exists. 2. Turn the apply pipeline off in `yodel.config.ts`: ```ts // yodel.config.ts export default defineConfig({ // ... ci: { forges: ["github", "gitlab", "forgejo"], apply: false }, }); ``` `apply: false` cannot be combined with a wave whose `approval` is `pr-review` or with `ci.resume`, since both run in the apply pipeline. `yodel ci` refuses either combination and names the setting to remove. 3. Render the pipelines with `npm run ci` (which runs `yodel ci`). With `apply: false` it writes `yodel-pr` and a `watch-` per environment on each forge, and no `yodel-waves.json`, no `yodel-apply` or `yodel-apply-plans` workflow, and no `.gitlab/yodel-apply.gitlab-ci.yml`; `.gitlab-ci.yml` has no apply stages and does not include that file. A project made from a template already has those files: `yodel ci` does not delete them, so delete them yourself and commit the result. `yodel ci --check` then passes, because it no longer expects them. 4. Add only the readers' secrets on the forge: for each environment, its URL and its reader (`DEV_CLICKHOUSE_URL`, `DEV_CLICKHOUSE_READER_USER`, `DEV_CLICKHOUSE_READER_PASSWORD` for `dev` on ClickHouse, `DEV_POSTGRES_...` on Postgres), and on GitLab `GITLAB_TOKEN`, a project access token with the `api` scope (Reporter) for the plan comment and the drift watch's issue. Where each one goes on each forge is in [Setting up each forge](/sql-yodeler/forges/). No writer's secret is needed. 5. Open a pull request with a migration and read the plan comment: the pending migrations for each environment, what they would run, and the digest an approval would be bound to. [The pull request comment](/sql-yodeler/approval/#the-pull-request-comment) shows one. ## What runs On each pull request, every job with readers or with no database credentials at all: - `lint`: `yodel ci --check`, `yodel lint`, and the replay check, which replays every migration into a throwaway server the job starts. It holds no database credentials. - `plan-`: `yodel config check --write-probe`, which tries a write and fails the job if the server allows it, then `yodel plan --comment`. It holds the environment's reader, and the reader of the environment before it in the waves (`waves[].requires`), so the plan can show where each migration has run. On its schedule, `watch-` compares the server with the declared schema and keeps a tracking issue open while it finds drift ([Drift](/sql-yodeler/drift/)). It holds the environment's reader. Applying is left to you. `YODEL_CREDENTIALS=writer yodel apply ` from a machine that holds the writer applies behind the same approval as the pipeline would ([Approving](/sql-yodeler/approval/#approving)), and records each migration in the history the plan comment and the watch read. ## Turning apply on later 1. Remove `apply: false` from `ci` in `yodel.config.ts`. 2. Run `npm run ci` and commit what it writes: `yodel-waves.json` and the apply pipeline on each forge. 3. Add the writers' secrets where [Setting up each forge](/sql-yodeler/forges/) says, so that only each environment's wave job can read them. The next push to main runs the apply waves, each waiting for its approval. [Approval](/sql-yodeler/approval/) covers the gates and the waves. ## Proven by The scenario claims below run what this page describes against a real server, once plain (it passes) and once with the behaviour broken (the claim catches it). [Claims status](/sql-yodeler/claims/) lists every claim. | Claim | What it says | Plain, broken | Last run | |---|---|---|---| | `template` | a project from the starter template, on Forgejo: apply only after approval, lint with replay and the plan comment on a pull request, and the approved change applied on merge; a sealed wave applies only on an approval sealed by a signer listed at the base, and a pr-review wave on the review of a writer other than the author; a pull request job cannot write, a forked migration fails lint and is annotated, a stale or hand-edited pipeline fails yodel ci --check, the CI image pinned by digest runs a pull request's jobs, a command token source mints the reader's password, and the drift watch keeps one tracking issue | ClickHouse: pass, caught; Postgres: pass, caught | `868ff97`, 2026-10-10 | | `template-github` | a project from the starter template, on GitHub Actions (act and a mock GitHub): apply only after approval, lint with replay and the plan comment on a pull request, and the approved change applied on merge; a sealed wave applies only on an approval sealed by a signer listed at the base, and a pr-review wave on the review of a writer other than the author; a pull request job cannot write, a forked migration fails lint and is annotated, a stale or hand-edited pipeline fails yodel ci --check, the CI image pinned by digest runs a pull request's jobs, a command token source mints the reader's password, and the drift watch keeps one tracking issue | ClickHouse: pass, caught; Postgres: pass, caught | `868ff97`, 2026-10-10 | | `template-gitlab` | | not recorded | | --- # Setting up each forge Source: https://intentius.io/sql-yodeler/forges/ ## Optional: hand this page to your coding agent ```text Prepare this repository's pipelines for the forges I name, following https://intentius.io/sql-yodeler/forges/: set `ci.forges` in yodel.config.ts, run `npm run ci`, and check that `npx yodel ci --check` passes. In the pull request description, list by name every secret, variable, environment, protection, token and runner the matrix says each forge needs. Put no secret's value in a file or in the description; I add them. Open a pull request with the result. Never run `yodel apply` against a shared environment, never run `chant approve`, `yodel approve` or `yodel override`, never edit the `chant/lifecycle` branch or `.chant/allowed_signers`, never merge; approvals and applies belong to people. ``` `yodel ci` (`npm run ci` in a project made from a starter template) renders the same jobs for GitHub Actions, GitLab CI and Forgejo Actions: lint and a plan comment per environment on each pull request, an apply wave per environment on a push to main, and a drift watch per environment on its schedule. Which job holds which credentials is the same on every forge ([Least privilege on each forge](/sql-yodeler/configuration/#least-privilege-on-each-forge)). What differs is where you put the secrets, which tokens the jobs use, and what the runner needs: | | GitHub | GitLab | Forgejo | |---|---|---|---| | Pipeline files | `.github/workflows/yodel-pr.yml`, `yodel-apply.yml`, `watch-.yml` | `.gitlab-ci.yml`, `.gitlab/yodel-apply.gitlab-ci.yml`, `.gitlab/yodel-watch.gitlab-ci.yml` | `.forgejo/workflows/yodel-pr.yml`, `yodel-apply.yml`, `watch-.yml` | | Readers' secrets | repository secrets | CI/CD variables, masked, not protected (merge request pipelines run on unprotected branches), with no environment scope | repository secrets | | Writer's secrets | environment secrets of the environment named ``, whose deployment branches are limited to `main`; only the wave job names that environment, so a pull request job cannot read them | CI/CD variables, protected, masked and scoped to the environment ``; only a job on a protected branch that declares that environment gets them, and only the wave job declares it | repository secrets: Forgejo has no environments, so a workflow on any branch can name them. Limit who can push branches to those trusted to apply | | The plan comment | the job's `GITHUB_TOKEN`, with `pull-requests: write` (the workflow grants it) | `GITLAB_TOKEN`: a project access token with the `api` scope (Reporter); `CI_JOB_TOKEN` cannot write merge request notes | the job's own token (`FORGEJO_TOKEN`, else `GITHUB_TOKEN`) | | The drift watch | `watch-.yml` on the watch's cron; the job's token with `issues: write` for the tracking issue | no cron in the file: one pipeline schedule per watch, with the cron and `CHANT_SCHEDULED_OP` noted at the top of `.gitlab/yodel-watch.gitlab-ci.yml`; `GITLAB_TOKEN` for the tracking issue | `watch-.yml` on the watch's cron; the job's own token writes the tracking issue | | Resuming after `yodel approve` | re-runs the run's failed jobs at the same commit; a fine-grained token with Actions: read and write | retries the job, and the waves after it run when it passes; a project access token with the `api` scope and the Developer role (`CI_JOB_TOKEN` cannot retry a job) | dispatches `yodel-apply.yml` again on the branch (no re-run API), so the waves before it plan again and apply nothing new; a token with `write:repository` | | Cloud roles over OIDC | the workflow grants `id-token: write` to a job that mints a cloud password | the job declares an ID token for the cloud's audience (`AWS_ID_TOKEN`, `GCP_ID_TOKEN` or `AZURE_ID_TOKEN`) | none: Forgejo issues no OIDC token to a job, so the runner's own cloud identity mints | | Runner requirements | `ubuntu-latest`; the lint job runs in a `node:22-bookworm` container with its replay server as a service, so a self-hosted runner needs Docker | a runner that runs `image:` and `services:` (the Docker executor, say); the jobs run in `node:22-bookworm` | an Actions runner with the `docker` label | | Pull requests from forks | they get no secrets and a read-only token: lint runs, but the plan jobs hold no reader and post no comment | a merge request from a fork runs its pipeline in the fork's project by default, with none of this project's variables | they get no secrets | `yodel approve` takes its token from `CHANT_FORGE_TOKEN`, else `GITHUB_TOKEN` or `GH_TOKEN` (GitHub and Forgejo), or `GITLAB_TOKEN` (GitLab), on your machine ([Approve and resume](/sql-yodeler/approval/#approve-and-resume)). With `ci: { apply: false }` ([Lint and plan on pull requests only](/sql-yodeler/lint-and-plan/)) there is no apply pipeline and no writer's secret on any forge, and the rows about the writer and resuming do not apply. A role whose password a cloud token source mints has no password variable; [Short-lived tokens](/sql-yodeler/configuration/#short-lived-tokens) has the sources, and `ci.login` for exchanging the forge's OIDC token. ## The steps on each forge The README a starter template writes into the project has the same steps, for reading offline. #### GitHub - Repository secrets: the readers' variables and the URLs (`DEV_CLICKHOUSE_URL`, `DEV_CLICKHOUSE_READER_USER`, `DEV_CLICKHOUSE_READER_PASSWORD` for `dev` on ClickHouse; `DEV_POSTGRES_...` on Postgres). - Environments `dev` and `prod` (Settings > Environments > ``), each with deployment branches limited to `main`, holding that environment's URL and writer variables. Only the wave job names the environment, so pull request jobs cannot read its secrets. Pull requests from forks get no secrets at all. - The workflows set their own token permissions: `contents: read` and `pull-requests: write` on `yodel-pr.yml` for the comment, `issues: write` on each `watch-.yml` for the tracking issue, and `contents: write` on `yodel-apply.yml`, which pushes the pending approval to `chant/lifecycle`. - The lint job runs in a `node:22-bookworm` container and reaches its replay server by the service's name, so it needs no free port on the runner; a self-hosted runner needs Docker. #### GitLab - CI/CD variables (Settings > CI/CD > Variables): the readers masked and not protected, with no environment scope, since merge request pipelines run on unprotected branches; the writers protected, masked and scoped to the environment `dev` or `prod`, which only the wave job for that environment declares. `.gitlab-ci.yml` lists the variables in its header. - `GITLAB_TOKEN`: a project access token with the `api` scope, for the plan comment and the watch's tracking issue (`CI_JOB_TOKEN` cannot write merge request notes or issues). For `yodel approve` to retry a job, the token needs the Developer role. - GitLab has no cron in the file: create one pipeline schedule per watch (Settings > CI/CD > Schedules) with the cron and the `CHANT_SCHEDULED_OP` value noted at the top of `.gitlab/yodel-watch.gitlab-ci.yml`. #### Forgejo - Repository secrets for all of the variables. Forgejo has no environments, so a workflow on any branch of the repository can name the writer's: restrict who can push branches to those trusted to apply, since a branch can change a workflow, or take changes only as pull requests from forks, which get no secrets. A token source whose cloud role trusts only the runner that runs `main` closes the rest. - The comment and the tracking issue are posted with the job's own token. - The jobs need an Actions runner with the `docker` label. ## Proven by The scenario claims below run what this page describes against a real server, once plain (it passes) and once with the behaviour broken (the claim catches it). [Claims status](/sql-yodeler/claims/) lists every claim. | Claim | What it says | Plain, broken | Last run | |---|---|---|---| | `template` | a project from the starter template, on Forgejo: apply only after approval, lint with replay and the plan comment on a pull request, and the approved change applied on merge; a sealed wave applies only on an approval sealed by a signer listed at the base, and a pr-review wave on the review of a writer other than the author; a pull request job cannot write, a forked migration fails lint and is annotated, a stale or hand-edited pipeline fails yodel ci --check, the CI image pinned by digest runs a pull request's jobs, a command token source mints the reader's password, and the drift watch keeps one tracking issue | ClickHouse: pass, caught; Postgres: pass, caught | `868ff97`, 2026-10-10 | | `template-github` | a project from the starter template, on GitHub Actions (act and a mock GitHub): apply only after approval, lint with replay and the plan comment on a pull request, and the approved change applied on merge; a sealed wave applies only on an approval sealed by a signer listed at the base, and a pr-review wave on the review of a writer other than the author; a pull request job cannot write, a forked migration fails lint and is annotated, a stale or hand-edited pipeline fails yodel ci --check, the CI image pinned by digest runs a pull request's jobs, a command token source mints the reader's password, and the drift watch keeps one tracking issue | ClickHouse: pass, caught; Postgres: pass, caught | `868ff97`, 2026-10-10 | | `template-gitlab` | | not recorded | | --- # Schema from an ORM Source: https://intentius.io/sql-yodeler/orm/ ## Optional: hand this page to your coding agent ```text Bring the tables our ORM defines into SQL Yodeler, following https://intentius.io/sql-yodeler/orm/: add a source under `sources` in yodel.config.ts with the command that prints the ORM's DDL, and remove any declaration in src/ of a table the ORM now owns. Run `npx yodel new ` and `npx yodel lint`, and open a pull request with the config, src/ and the new migration. Never run `yodel apply` against a shared environment, never run `chant approve`, `yodel approve` or `yodel override`, never edit the `chant/lifecycle` branch or `.chant/allowed_signers`, never merge; approvals and applies belong to people. ``` When an ORM defines some of the tables, yodel can read them from the DDL the ORM prints instead of having them rewritten as declarations. The ORM's tables then go through the same plan, approval and apply as the ones in `src/`. ## Plain .sql files in src/ The simplest way to keep schema as DDL is a `.sql` file in `src/`. chant's build reads every `.sql` file in the source directory beside the TypeScript declarations, so yodel needs no configuration for it: ```sql -- src/billing.sql CREATE TABLE app.invoices ( id bigint PRIMARY KEY, customer_id bigint NOT NULL REFERENCES app.customers (id), amount numeric(12, 2) NOT NULL ); CREATE INDEX invoices_customer ON app.invoices (customer_id); ``` A file reads the same statements a DDL source does (below). A name in it that another object declares, in a `.sql` file or in TypeScript, is a dependency, so the plan creates `app.customers` first. An object declared in a file and in a template is a build error naming both. A file whose first line is `-- chant-discovery-skip` is not read, for seed data or queries kept beside the schema. Use a file when you write or paste the DDL yourself. Use a DDL source when a command prints it, such as an ORM's, so the declarations follow the model each time yodel builds. ## A DDL source Name each source in `yodel.config.ts` under `sources`, with the command that prints its DDL or a file that holds it: ```ts import { defineConfig } from "@intentius/sql-yodeler"; export default defineConfig({ sources: { prisma: { command: "npx prisma migrate diff --from-empty --to-schema-datamodel prisma/schema.prisma --script", schema: "app", }, legacy: { file: "db/legacy.sql" }, }, }); ``` | Setting | What it is | |---|---| | `command` | a shell command, run in the project directory, that prints the DDL on stdout | | `file` | a file of DDL, relative to the project directory | | `schema` | the Postgres schema (ClickHouse database) for the names the DDL leaves unqualified, as an ORM does when its connection picks the schema | Give one of `command` and `file`. A source's name is lowercase letters, digits and `-`. Commands that print an ORM's DDL: | ORM | Command | |---|---| | Prisma | `npx prisma migrate diff --from-empty --to-schema-datamodel prisma/schema.prisma --script` | | Drizzle | `npx drizzle-kit export` (or `drizzle-kit generate`, then a `file` source on the SQL it wrote) | | Django | `python manage.py sqlmigrate `, one source per app | | SQLAlchemy | a script that prints `CreateTable(table).compile(engine)` for each table of the metadata | ## What yodel does with it Before each build (`yodel plan`, `yodel new`, `yodel apply`, `yodel drift`), yodel runs each source's command or reads its file, parses the DDL with the sql lexicon, and writes it out as declarations in `src/.generated.ts`: the same file `chant import` writes, with a header saying where it came from. The build reads that file like any other declaration, so a model change shows up in `yodel plan` as the change it makes to the table, and the ApplyOp's own build finds it too. Commit the generated file. A build that yodel does not start, such as `chant build` or an Op run outside yodel, reads the committed copy. Keep a `file` source's DDL out of the schema build. chant's build reads every `.sql` file in the directory it builds, and an ApplyOp's diff builds the project root unless `chant.config.ts` sets `sourceDir`. A DDL file there would declare each object a second time, beside the generated file. Set `sourceDir: "src"` in `chant.config.ts`, or start the file with a `-- chant-discovery-skip` line. yodel reads the DDL with the sql lexicon's `sqlFileDeclarations`, the reader chant's build uses for a `.sql` file in `src/`, so a source and a file read the same statements. On Postgres that is each `CREATE`, `GRANT`, `REVOKE` and `ALTER DEFAULT PRIVILEGES`, with `COMMENT ON` and `ALTER TABLE ... ROW LEVEL SECURITY` folded into the object they finish. ORMs print foreign keys as `ALTER TABLE ADD CONSTRAINT ... FOREIGN KEY ...`; each `ALTER TABLE ... ADD` of a table constraint is folded into ``'s `CREATE TABLE`, where a declaration holds it. `BEGIN` and `COMMIT` are skipped. On ClickHouse, it reads `CREATE DATABASE`, `TABLE`, `VIEW`, `MATERIALIZED VIEW`, `DICTIONARY` and `FUNCTION`. Any other statement is an error that names it, and nothing is built. Leaving it out would leave part of the ORM's schema unmanaged without saying so. With `schema` set, the unqualified names are qualified with it: on Postgres, each object's own name, an index's table, a foreign key's table, and a column whose type is an enum or domain the same DDL creates; on ClickHouse, each object's own name except a function's, which belongs to no database. ## One owner per object An object is declared in one place. If a source creates a table that `src/` (or another source) also declares, the build stops and names both: ```text $ yodel plan dev yodel plan: each object has one owner, and app.customers is declared twice: by src/ (export customers) and by the source "prisma" (src/prisma.generated.ts). Leave it to one of them. ``` The same holds when `src/` exports the object under another name: yodel reads each built object's name and refuses the build when a source creates it too. ## Proof The `declarative` claim runs a project whose `yodel.config.ts` has a source whose command prints a Prisma-style DDL file (the claim installs no ORM): the first apply creates the ORM's tables behind the gate, a column added to the model is the one change `yodel plan` shows, the approved apply adds it, and the plan is then empty. Under `BREAK=1` the DDL changes again after the approval, and the column is not applied. ## Proven by The scenario claims below run what this page describes against a real server, once plain (it passes) and once with the behaviour broken (the claim catches it). [Claims status](/sql-yodeler/claims/) lists every claim. | Claim | What it says | Plain, broken | Last run | |---|---|---|---| | `declarative` | the declarative path: yodel plan shows the change against the live server and yodel apply makes it behind the plan-bound gate, for src/ declarations (on Postgres, functions, procedures and triggers, and access control: a role, a policy and grants, with a grant made by hand revoked; on ClickHouse, a dictionary, a function, and access control: a role, a user, a row policy and grants, with a grant made by hand revoked) and for an ORM's exported DDL | ClickHouse: pass, caught; Postgres: pass, caught | `c6f58a4`, 2026-10-10 | --- # Generated schema reference Source: https://intentius.io/sql-yodeler/schema-docs/ ## Optional: hand this page to your coding agent ```text Following https://intentius.io/sql-yodeler/schema-docs/, run `npx yodel docs` and open a pull request with schema-docs/. In its description, name the tables and relations the diagram shows that the pull request changes. Never run `yodel apply` against a shared environment, never run `chant approve`, `yodel approve` or `yodel override`, never edit the `chant/lifecycle` branch or `.chant/allowed_signers`, never merge; approvals and applies belong to people. ``` This page is about the reference `yodel docs` generates. Writing the schema itself is [Declaring the schema](/sql-yodeler/schema/). `yodel docs` writes a reference of the schema for people reading a pull request or onboarding to a project: one HTML page and one Mermaid entity-relationship diagram, from the build of the declared schema. It reads no server and starts nothing; the HTML has its styles inline and no scripts, so it opens from disk or from any static host. ```sh npx yodel docs # writes schema-docs/index.html and schema-docs/erd.md npx yodel docs --out site/schema # somewhere else npx yodel docs --migration 20261010T1723-baseline # the schema that migration recorded ``` `schema-docs/index.html` lists every database or schema, table, view, materialized view and index, with: - its columns: type, nullability, default or generated expression, codec, and comment - its keys: the primary key, unique constraints, checks and foreign keys on Postgres; the engine with its arguments, `ORDER BY`, `PRIMARY KEY`, `PARTITION BY`, `SAMPLE BY` and `TTL` on ClickHouse - its relations both ways: the foreign keys it holds and the ones that reference it, what a view or materialized view reads, the table a materialized view writes to, the local table a `Distributed` table serves - its history: each step in `migrations/` that changed it, in chain order - its DDL `schema-docs/erd.md` is the diagram as a Markdown file with a `mermaid` block, which GitHub, GitLab and Forgejo render in place. A foreign key is drawn as many-to-one (zero-or-one when its columns may be null); the other relations are dashed. Columns in a primary key are marked `PK`: on ClickHouse that is the `PRIMARY KEY`, or the `ORDER BY` when the table sets none, since the sparse index is built on it. Mermaid takes a type as one word, so spaces and commas in a type become `_` in the diagram; the HTML shows the type as declared. The same schema always gives the same files, so a project can commit them and a pull request that changes the schema shows the change in `erd.md` and `index.html` next to the migration. `--migration ` documents the schema a migration recorded instead of the build, with the history up to that migration. ## The examples Both example projects commit their output, and `npm run check` fails when it is not what `yodel docs` writes now (`test/schema-docs.test.ts`). The ClickHouse example declares one database and one table. [Its reference](/sql-yodeler/examples/clickhouse/schema/) shows the table's `MergeTree` engine, its sort and partition keys, and the migrations that changed it, the sort-key rebuild among them. A backfill step names no object, so it is in no history. 
erDiagram
  shop_events["shop.events"] {
    UInt64 id PK
    LowCardinality(String) kind PK
    DateTime at
    LowCardinality(String) country
    LowCardinality(String) source
  }
The Postgres example declares a schema, a table and three indexes. [Its reference](/sql-yodeler/examples/postgres/schema/) shows the primary key, the check constraint, the indexes, and the history, the column rename as an expand-and-contract step included. 
erDiagram
  shop_orders["shop.orders"] {
    bigint id PK
    text email
    text status
    numeric(12_2) amount
    timestamp_with_time_zone placed_at
    timestamp_with_time_zone refunded_at
    boolean gift
  }
Neither example has a foreign key or a view, so neither diagram has a relation line. A project with them gets one per foreign key, per table a view reads and per materialized view target. --- # The two workflows Source: https://intentius.io/sql-yodeler/workflows/ ## Optional: hand this page to your coding agent ```text Make the schema change I describe with SQL Yodeler, following https://intentius.io/sql-yodeler/workflows/: edit the declarations in src/, run `npx yodel new ` and `npx yodel lint`, and open a pull request with src/ and the new migration. Do not edit a migration that is committed on the default branch; write a new one. Never run `yodel apply` against a shared environment, never run `chant approve`, `yodel approve` or `yodel override`, never edit the `chant/lifecycle` branch or `.chant/allowed_signers`, never merge; approvals and applies belong to people. ``` Both workflows start from the schema declared in `src/` ([Declaring the schema](/sql-yodeler/schema/)): it says what the database should be, and yodel works out how to get there. `yodel plan ` and `yodel apply ` take one of two paths, chosen by the project: the versioned path when it has a `migrations/` directory holding at least one migration, the declarative path otherwise. (A template's empty `migrations/`, with only `.gitkeep`, is still declarative.) Both apply behind an approval bound to a digest of what they would do; see [Approval](/sql-yodeler/approval/). ## Declarative: the declared schema against the live database No migration files. `yodel plan ` shows the classified change from the environment's live database to the declared schema in `src/`, as `chant sql plan` reports it, and `yodel apply ` makes it. `yodel apply` plans, then runs the project's `ApplyOp` for the environment with `chant run`. From the ClickHouse example: ```ts import { ApplyOp } from "@intentius/chant/op"; // The declarative path: yodel apply dev plans src/ against dev and runs this // Op, behind a gate bound to the plan. export const { op } = ApplyOp({ name: "apply-dev", env: "dev", target: "clickhouse", gate: {} }); ``` The Op builds the project first with `npm run build`, so `package.json` needs a `build` script that writes chant's build output: ```json "scripts": { "build": "chant build src --lexicon sql -o dist/schema.json" } ``` Without it the Op fails at its Build step (`npm error Missing script: "build"`). The run below is the ClickHouse example's first apply. The Op stops at its gate and `yodel apply` exits 3 with the command that approves this plan. Today that command is `npx yodel approve dev --plan `, which shows the plan, asks you to type `dev`, and records the approval of the digest the gate waits on; the runs below were recorded before yodel printed it, so they show and run chant's `chant approve`, which records the same approval: ```console $ npx yodel apply dev Plan for dev (clickhouse 26.8.15.10 at 127.0.0.1:8123, sql.profiles.dev) shop [create] object: shop SQLCH200 Create an object. A new database, table, view, dictionary, function, user, role or row policy is created, or a grantee gets its first grants; nothing existing changes. https://clickhouse.com/docs/sql-reference/statements/create events (shop.events) [create] object: shop.events SQLCH200 Create an object. A new database, table, view, dictionary, function, user, role or row policy is created, or a grantee gets its first grants; nothing existing changes. https://clickhouse.com/docs/sql-reference/statements/create 2 create Access control: not managed in dev (off by default); policies, grants, roles and users are not planned. Running apply-dev (/clickhouse/ops/apply-dev.op.ts) > sql-yodeler-example-clickhouse@0.0.0 build > chant build src --lexicon sql -o dist/schema.json fold: 1 file folded, 0 ran sql — environment: dev 2 missing, 0 orphan, 0 disappeared, 0 newly observed, 0 drifted, 0 unchanged -------------------------------------------------------------------------------- MISSING (declared, provider reports not in cloud): - events [queried http://127.0.0.1:8123 shop.events] [definition 9f20bfc4d4ef488a683af5646837a026beea36f8affb953ecfb1e118c3f4a21a] - shop [queried http://127.0.0.1:8123 shop] [definition aecfb2aa109301003bbc1a7ee057c2bb2610d824e610588317907171ed70c610] [phase] Build ✓ chantBuild(path=.) 728ms [phase] Plan ✓ lifecycleDiff(env=dev, live=true) 747ms [outcome] Drift=true [phase] Approve • gate:approve-apply-dev() skipped [phase] Apply • nativeApply(target=clickhouse, env=dev, output=dist/schema.json, deleteMode=never) skipped Op "apply-dev" is gated on "approve-apply-dev" after 2.0s Approve apply to dev (delete mode: never) plan : jcs1-sha256:02cfabc13e5ef7d0f1d089b4a5468afc3ed5140b9c2f04afad2203b470c933e8 approve : chant approve apply-dev approve-apply-dev --plan jcs1-sha256:02cfabc13e5ef7d0f1d089b4a5468afc3ed5140b9c2f04afad2203b470c933e8 expires : 2026-10-12T17:21:58.086Z apply-dev is waiting at gate "approve-apply-dev" for approval of this plan. Approve it with: chant approve apply-dev approve-apply-dev --plan jcs1-sha256:02cfabc13e5ef7d0f1d089b4a5468afc3ed5140b9c2f04afad2203b470c933e8 then run npx yodel apply dev again. [exit 3] ``` ```console $ npx chant approve apply-dev approve-apply-dev --plan jcs1-sha256:02cfabc13e5ef7d0f1d089b4a5468afc3ed5140b9c2f04afad2203b470c933e8 Gate "approve-apply-dev" on "apply-dev" resolved by yodel at 2026-10-10T17:21:59.886Z This approves the plan jcs1-sha256:02cfabc13e5ef7d0f1d089b4a5468afc3ed5140b9c2f04afad2203b470c933e8, and only that plan. A run whose fresh plan differs refuses rather than applying it. This records the resolution as a fact; it does not itself re-run anything. Run `chant run apply-dev` and it walks through gate "approve-apply-dev". ``` Run again, the same command applies the plan; the example shows that run, which ends `Applied: dev matches the declared schema.` What the declarative path does and does not do: - An `ApplyOp` with no `delete` option runs with delete mode `never` (its output says `delete mode: never`): a drop stays in the plan, and `yodel apply` says so when changes remain after the Op. `delete: "owned-only"` or `"gated"` on the `ApplyOp` lets it drop (chant's `ApplyOp` options). - A change no statement makes in place (a ClickHouse sort-key change, a Postgres column rename or type change across kinds) makes `yodel plan` exit 2, and `yodel apply` applies nothing. Those changes are steps of a migration on the versioned path ([Data migrations](/sql-yodeler/steps/)). - It keeps no history. What the database is, is what the live catalog says; `yodel drift` compares the two ([Drift](/sql-yodeler/drift/)). - The topology comes from `sql.profiles..topology` only ([Topology](/sql-yodeler/topology/)). - chant marks the objects it creates with an ownership marker in their comment (`[chant managed-by=chant]`) and never touches an object the project does not declare. ## Versioned: migrations written from the declared schema The declared schema stays the source. `yodel new ` writes the next migration by diffing it against the schema the newest migration recorded, with no database involved. Each change goes through review as a pull request holding the edit to `src/` and the migration it produced: ```sh $EDITOR src/schema.ts npx yodel new add-country # writes migrations/-add-country/ npx yodel lint # offline; CI runs it on every pull request npx yodel plan dev # the pending migrations and the plan digest npx yodel apply dev # behind the migrations Op's gate ``` `yodel apply` on this path runs the pending migrations in chain order, statement by statement, recording each in the history table under a lock, and resumes a migration that failed part way at the statement that failed. A change that is not one statement (a rebuild, a backfill, an expand-and-contract column change) is a step inside the migration, recorded in the same history. [Migrations](/sql-yodeler/migrations/) has the details, and [Data migrations](/sql-yodeler/steps/) the steps. ## More than one environment Each environment is a profile in `chant.config.ts`, with its own history. The starter templates' apply pipeline applies them in the order of `yodel.config.ts` `waves` after every push to main, one job per environment, each waiting for the one before; a wave that stops at its gate stops the ones after it. Each wave has its own gate policy (`always`, `on-destructive` or `never`), read from the commit the push merged onto, and applies only migrations the environment before it has applied: prod refuses (exit 4) a migration staging's history does not record. [Approval](/sql-yodeler/approval/#waves-a-gate-per-environment) has how a wave plans, waits and applies. ## The pipelines: yodel ci `yodel ci` renders a project's pipelines for GitHub Actions, GitLab CI and Forgejo Actions from `chant.config.ts` (the environments), `yodel.config.ts` (`waves`, `ci`) and `ops/` (each environment's migrate and watch Ops), each declared with chant's lexicons: `yodel-pr` (lint, the replay check and the plan comment on a pull request), `yodel-apply` (the waves, after a push to main) and a `watch-` per environment. The templates' `npm run ci` runs it. [Setting up each forge](/sql-yodeler/forges/) has the secrets, tokens and runner each forge needs for them. `ci: { apply: false }` leaves out `yodel-apply` and `yodel-waves.json` on every forge, for a team that wants lint and the plan comment on pull requests before CI applies anything: [Lint and plan on pull requests only](/sql-yodeler/lint-and-plan/). The pipelines come from the installed package, so a fix to them reaches a project when it upgrades `@intentius/sql-yodeler` and runs `yodel ci` again. `yodel ci --check` writes nothing and exits 1 naming each file that differs from what it renders; the pull request's lint job runs it, so a pipeline edited by hand, or not rendered again after an upgrade, fails the pull request until the rendered files are committed. `yodel ci --replay` runs the lint job's replay check on your machine, in a throwaway server it starts with Docker. `ci.forges` in `yodel.config.ts` picks the forges (default all three). A project's own jobs go in the module `ci.jobs` names, declared with chant, never in the rendered files: ```ts // ci/jobs.ts, with ci: { jobs: "ci/jobs.ts" } in yodel.config.ts import { Job, Step } from "@intentius/chant-lexicon-github"; import { Image, Job as GitLabJob } from "@intentius/chant-lexicon-gitlab"; export const actions = { test: new Job({ "runs-on": "ubuntu-latest", steps: [new Step({ uses: "actions/checkout@v7" }), new Step({ run: "npm ci && npm test" })] }), }; export const gitlab = { test: new GitLabJob({ stage: "test", image: new Image({ name: "node:22" }), script: ["npm ci", "npm test"] }), }; ``` `yodel ci` adds `actions` to `yodel-pr` on GitHub and Forgejo and `gitlab` to `.gitlab-ci.yml`, after its own jobs, with a GitLab job's stage after `review`, so they are there after every render. A job with the name of one of yodel's (`lint`, `plan-`, `.yodel`) is refused. By default every job runs in `node:22-bookworm` (GitHub's plan jobs on the runner, with `setup-node`) and installs the project's packages with `npm ci`. `ci.image` runs every job in the CI image instead, `ghcr.io/intentius/sql-yodeler:@sha256:`, which a release publishes with Node, yodel, chant and the lexicons on it: no job runs `npm ci` or `setup-node`, and a checkout resolves chant and the lexicons from the image. It is taken only pinned by its digest, which the release's `image` job writes to its summary, so a job never runs a different image under the same tag. Use the version the project's `package.json` installs, so the image's yodel renders the same pipelines `yodel ci --check` compares; a project whose jobs need packages of their own leaves `image` unset. ## ClickHouse objects A ClickHouse project declares these with the `sql` lexicon's tags, from `@intentius/chant-lexicon-sql/clickhouse`. Both paths plan and apply them, `yodel drift` watches them, and `yodel init --from` adopts them. The claims column names the scenario claims that run each against a server. | Object | Tag | Claims | | --- | --- | --- | | Database | `database` | declarative, new, adopt | | Table | `table` | declarative, new, adopt | | View and materialized view | `view` | adopt | | Dictionary | `dictionary` | declarative, adopt | | Function (a SQL lambda) | `func` | declarative, adopt | | User | `user` | declarative, adopt | | Role | `role` | declarative, adopt | | Row policy | `policy` | declarative, adopt | | Grant | `grant` | declarative, adopt | Dictionaries and functions: - A dictionary's changed comment is SQLCH203. Any other change (its source, layout, lifetime, attributes) is SQLCH245, applied as one `CREATE OR REPLACE DICTIONARY`. - A function belongs to no database. A changed lambda is SQLCH260, applied as `CREATE OR REPLACE FUNCTION`; the server prints `x * k` as `(x * k)`, and the plan asks the server's formatter before calling that a change. A function cannot carry chant's ownership marker, so it is never pruned and a function made by hand is never read. `yodel init --from` adopts the SQL functions `sql.profiles..importFunctions` names (names, or prefixes ending in `*`), and the functions those call; it warns naming each function on the server it leaves out. It also adopts a function an adopted view or column default calls: ClickHouse stores the function's body in place of the call, and chant's import matches that body to the function, so a function in use needs no entry in the list. - Users, roles, row policies and grants are planned only where the environment manages access ([Access control](/sql-yodeler/access/#clickhouse)). ## Postgres objects A Postgres project declares these with the `sql` lexicon's tags, and both paths plan, apply, watch for drift and adopt them. The claims column names the scenario claims that run each against a server. | Object | Tag | Claims | | --- | --- | --- | | Schema | `schema` | declarative, new, adopt | | Table | `table` | declarative, new, adopt | | Index | `index` | declarative, new, adopt | | Function | `func` | declarative, new, adopt | | Procedure | `procedure` | declarative, new | | Trigger | `trigger` | declarative, new, adopt | | View and materialized view, sequence, enum, domain, extension | `view`, `sequence`, `type`, `domain`, `extension` | none yet | Functions, procedures and triggers: - A function or procedure is its name and its parameter types, so two overloads are two objects. A new body or new attributes are `CREATE OR REPLACE` (SQLPG280, metadata). A change `CREATE OR REPLACE` refuses (the result type, an output parameter, an input parameter's name, a removed default, or other parameter types between two builds) is a drop and a create in one transaction (SQLPG281); while a view, trigger, default or other routine depends on the function it is an expand change (SQLPG282), which `yodel new` writes as a manual step. - A trigger is its name on its table. Created on a table that already exists, it takes SHARE ROW EXCLUSIVE on that table (SQLPG283, metadata); created with its table, it is part of the create (SQLPG200). A changed trigger is `CREATE OR REPLACE TRIGGER` (SQLPG284), and a dropped one is SQLPG285. - Statements go in dependency order: a function before the trigger that runs it and after the tables its body names, and a trigger dropped before its function. - `yodel lint` flags a migration that drops and creates a function or procedure with another signature (`pg-signature`), and one that drops a function a declared trigger still executes (`pg-routine-in-use`); see [Lint](/sql-yodeler/lint/). - `yodel drift` reports a function replaced by hand and a declared trigger that is gone. `yodel init --from` adopts the functions, procedures and triggers in the profile's schemas with the tables. - On the declarative path, a function or trigger on the server that the project does not declare shows in `yodel plan` as a drop, the way an undeclared table does. Only a prune makes it, and only for an object carrying chant's ownership marker. A migration never drops one unless an earlier migration declared it. - A routine with a SQL-standard body (`RETURN ...`, `BEGIN ATOMIC`) cannot be declared: the server stores it rewritten. `init --from` leaves one out with a warning. ## Moving from one to the other A project on the declarative path moves to the versioned one with `yodel init --from `: it imports the live database into `src/`, writes the baseline migration that creates all of it, and records the baseline applied in the environment's history without running it. With `src/` already declaring the schema, `--force` lets it write over the declarations. Then the `ApplyOp` gives way to a migrations Op. Both examples start declarative and move this way; [Starting a project](/sql-yodeler/adoption/) has the run. Going back is deleting `migrations/` and adding an `ApplyOp`. The history table stays in the database, unread. ## Proven by The scenario claims below run what this page describes against a real server, once plain (it passes) and once with the behaviour broken (the claim catches it). [Claims status](/sql-yodeler/claims/) lists every claim. | Claim | What it says | Plain, broken | Last run | |---|---|---|---| | `declarative` | the declarative path: yodel plan shows the change against the live server and yodel apply makes it behind the plan-bound gate, for src/ declarations (on Postgres, functions, procedures and triggers, and access control: a role, a policy and grants, with a grant made by hand revoked; on ClickHouse, a dictionary, a function, and access control: a role, a user, a row policy and grants, with a grant made by hand revoked) and for an ORM's exported DDL | ClickHouse: pass, caught; Postgres: pass, caught | `c6f58a4`, 2026-10-10 | | `new` | yodel new writes the next migration offline, with no dev database, and it applies; on Postgres, functions, procedures and triggers too, which lint checks | ClickHouse: pass, caught; Postgres: pass, caught | `c6f58a4`, 2026-10-10 | | `approval` | a pending migration applies only after chant approve of its plan digest, and the history records that digest; a failing pre-migration check refuses it, and so does a policy rule, read at the base commit, unless an override is recorded for that digest, for a yodel revert as for an apply; the audit log derived from the history and the ledger accounts for every approval and apply, and names one removed; a reader and a writer that are one user are refused, and the reader, its password a minted token, cannot write; an approval is refused once the migration changed after it, and nothing is applied | ClickHouse: pass, caught; Postgres: pass, caught | `c6f58a4`, 2026-10-10 | | `waves` | the apply pipeline runs one wave per environment, in order, each behind its gate policy read from the base commit, applies only what the wave before applied, and applies a tenant set's migrations to every tenant behind one gate; a sealed wave counts only an approval sealed by a signer the base commit lists | ClickHouse: pass, caught; Postgres: pass, caught | `c6f58a4`, 2026-10-10 | | `adopt` | yodel init --from adopts a live database without touching it, and yodel plan then shows no change; on Postgres its policies, row-level security and grants too, on ClickHouse its dictionaries, functions, roles, users, row policies and grants; yodel init --baseline records the baseline, behind the gate, in a second environment that holds the same schema, and refuses one that differs | ClickHouse: pass, caught; Postgres: pass, caught | `9329873`, 2026-10-10 | --- # Migrations Source: https://intentius.io/sql-yodeler/migrations/ ## Optional: hand this page to your coding agent ```text `yodel lint` reports a fork on this branch. Following https://intentius.io/sql-yodeler/migrations/ (Forks and yodel rebase), run `npx yodel rebase` with `--env` for each environment I name, so a migration that ran is never moved, then `npx yodel lint`, and push the branch. Never run `yodel apply` against a shared environment, never run `chant approve`, `yodel approve` or `yodel override`, never edit the `chant/lifecycle` branch or `.chant/allowed_signers`, never merge; approvals and applies belong to people. ``` On the versioned path, `migrations/` holds the history of the schema as reviewed steps, and a table in each environment's database records what ran there. This page covers both, how `yodel apply` uses them, and what to do when they disagree. The output quoted here is from the two examples' runs; `...` marks lines of a command's output left out. ## The directory Each migration is a directory, `migrations/-/`, named by `yodel new ` with the UTC time it was written. The name is lowercase letters and digits, words joined by `-` or `_`. It holds: - `migration.sql`: the statements, each after a comment naming its object, the classifier's rule and the change's class. A step that is not SQL (a rebuild, a backfill, an expand-and-contract change, a manual step) appears as a comment only. - `migration.json`: the migration's id, its parent's id, its checksum, when it was written, its dialect, its steps (each statement with its SQL, rule and class, and each Op step with the Op's declaration and options), and the recorded schema: chant's build of the declared schema as it stood after this migration. - any file a step names: a backfill written as a chant Op, `backfill.ts`. A backfill written as a form has no file of its own; the form is in the step in `migration.json`. From the ClickHouse example: ```sql -- yodel migration 20261010T1722-add-country -- parent: 20261010T1722-baseline -- events (shop.events): SQLCH201 metadata ALTER TABLE `shop`.`events` ADD COLUMN country LowCardinality(String) DEFAULT '' AFTER `at`; ``` The order migrations run in is the chain of parent ids. The timestamp in an id is for people reading a listing; it never decides the order. The first migration has no parent (`"parent": null`). ### Checksums A migration's checksum covers its own files and nothing else: `migration.sql` with its line endings normalized, `migration.json` as parsed JSON without its `checksum` field (so whitespace and key order do not count, but every value does, the parent and the recorded schema included), and every other file in its directory. Adding, editing or removing another migration never changes it. Two branches that each add a migration therefore never conflict over a shared file; they fork the chain instead, which `yodel lint` reports ([below](#forks-and-yodel-rebase)). `yodel lint` checks each migration's files against its `checksum` field (rule `checksum`). `yodel status` and `yodel apply` check each applied migration's files against the checksum the history recorded when it ran, and refuse on a mismatch. ## Writing one: yodel new `yodel new ` diffs the newest migration's recorded schema (an empty schema for the first) against a fresh build of `src/`, and writes the next migration. It reads no database. When nothing changed it writes nothing and exits 0: ```console $ npx yodel new check No changes: the declared schema matches the newest migration's recorded schema (20261010T1724-rename-email). Nothing written. ``` (From the Postgres example's run, just after it writes `rename-email`; `run.ts postgres --write` rewrites this quote with the README. The committed-example check in `examples/walkthrough/run.ts` runs `yodel new check` in each example, so a declared schema that drifts from its migrations fails the e2e test.) It refuses (exit 1) when the directory is not one chain (a fork, a missing parent) or a migration of that id is already there. `yodel new --backfill` ends the migration with a backfill step written from `backfills/.json` ([Data migrations](/sql-yodeler/steps/#backfills)), and `--retain ` keeps the old column of a Postgres rename, or the old table of a rebuild, that long after the switch ([Data migrations](/sql-yodeler/steps/#postgres-postgresmigrationop)). ## The history table Each environment's history is a table in its own database: `.history` on ClickHouse, `.history` on Postgres, where the history database (schema) is `yodel.config.ts`'s `environments..history.database`, else `YODEL_HISTORY_DATABASE`, else `yodeler`. `yodel apply` (or `yodel init`, or `yodel repair`) creates it when it is not there. It is never part of the declared schema. The plan digest covers what it records (the applied migrations and their checksums) but not the table itself, so recording a run does not move the digest. It is append-only: one row per event, never updated or deleted. | Column | What it holds | |---|---| | `seq` | the event's order: taken under the apply lock, one more than the highest, so it orders rows across runs without trusting a clock | | `run_id` | the `yodel apply` run that wrote the row | | `kind` | `migration` (a migration started, succeeded or failed; `statement_index` -1), `statement`, or `step` (an Op step) | | `migration_id`, `parent`, `checksum` | the migration, its parent, and the checksum its files had | | `statement_index`, `statement_sha` | which step of the migration, and the hash of the statement as written (before topology rendering) | | `status` | `started`, `succeeded` or `failed` | | `error` | why it failed | | `applied_by` | who ran it (`yodel@example` in the examples) | | `plan_digest` | the digest the run was approved under | | `silences` | the migration's `yodel:allow` silences, as JSON ([Lint](/sql-yodeler/lint/#silencing-a-finding)) | | `note` | a step's summary as JSON; a statement's topology (`topology=single`); `baseline: ...` on a baseline `yodel init` recorded; a repair's reason | | `recorded_at` | the server's time, for people | The state of a migration or a statement is its row with the highest `seq`. The ClickHouse example's rows for its rebuild and its backfill: ```sql SELECT migration_id, kind, statement_index, status, note, silences FROM yodeler.history WHERE migration_id LIKE '%events-by-id' OR migration_id LIKE '%fill-country' ORDER BY seq FORMAT Vertical ``` ``` Row 1: ────── migration_id: 20261010T1722-events-by-id kind: migration statement_index: -1 status: started note: silences: [{"step":0,"rule":"ch-rebuild","reason":"600 rows; the copy takes seconds"}] Row 2: ────── migration_id: 20261010T1722-events-by-id kind: step statement_index: 0 status: started note: silences: [{"step":0,"rule":"ch-rebuild","reason":"600 rows; the copy takes seconds"}] Row 3: ────── migration_id: 20261010T1722-events-by-id kind: step statement_index: 0 status: succeeded note: {"op":"ClickHouseRebuildOp","table":"shop.events","state":"rebuild","partitions":6,"copied":6,"skipped":0,"cleared":0,"repaired":0,"verifiedRows":600,"swapped":true,"oldTable":"shop.events__chant_old","retainUntil":"2026-10-17T17:22:47.304Z"} silences: [{"step":0,"rule":"ch-rebuild","reason":"600 rows; the copy takes seconds"}] Row 4: ────── migration_id: 20261010T1722-events-by-id kind: migration statement_index: -1 status: succeeded note: silences: [{"step":0,"rule":"ch-rebuild","reason":"600 rows; the copy takes seconds"}] Row 5: ────── migration_id: 20261010T1722-fill-country kind: migration statement_index: -1 status: started note: silences: [] Row 6: ────── migration_id: 20261010T1722-fill-country kind: step statement_index: 0 status: started note: silences: [] Row 7: ────── migration_id: 20261010T1722-fill-country kind: step statement_index: 0 status: succeeded note: {"op":"backfill","file":"backfill.ts","name":"backfill-fill-country","effects":6,"skipped":0,"ran":6} silences: [] Row 8: ────── migration_id: 20261010T1722-fill-country kind: migration statement_index: -1 status: succeeded note: silences: [] ``` ## The apply lock One apply runs at a time per environment. - ClickHouse: a row in `.lock`, a `KeeperMap` table, so it lives in Keeper and every runner on any host sees it. It needs Keeper and the server setting `keeper_map_path_prefix`. The holder renews it while it works; a lock left by a dead runner expires after its TTL (`lockTtl`, `YODEL_LOCK_TTL`, default 900 seconds). A single node without Keeper gets a lock file on the runner's machine instead, which serializes applies from that machine only, and the run says so (`lock: no KeeperMap on this server ...; the lock is a file on this machine, so cross-runner locking needs Keeper`). A cluster, a `Replicated` database and Cloud refuse rather than fall back. - Postgres: a session advisory lock on a connection of its own, keyed by the history schema (`lock: held in Postgres advisory lock (...) for history schema yodeler`). Postgres releases it when the session ends, however it ends, so it needs no expiry. A run that finds the lock taken exits 5 and names the holder, unless it was told to wait: `yodel apply --lock-wait 10m` (or `lockWait` in seconds in `yodel.config.ts`, or `YODEL_LOCK_WAIT`) tries again every second for up to that long, saying who holds the lock when the wait starts and whenever the holder changes, and exits 5 only if the lock is still held at the end. The wait is in the Apply step's JSON (`lock.waited`: how long, and who held it). An apply that gets the lock after another one applied everything finds nothing pending under it and sends nothing. `yodel apply --stand-down` (or `YODEL_STAND_DOWN=1`) is for CI on the main branch, where two merges in quick succession start two apply jobs: before it takes the lock, and while it waits for it, the run fetches its branch from `origin` (`GITHUB_REF_NAME` on GitHub and Forgejo Actions, `CI_COMMIT_BRANCH` on GitLab CI, else the checked-out branch), and when the branch's tip is a newer commit that contains this one, it applies nothing and exits 0 (`standing down: main on origin is at , newer than this run's , and its own apply applies `; `"status": "stood-down"` in the JSON). The newer commit's apply job applies the same environment. A fetch that fails, or a tip that does not contain the commit (a force-push), is not a newer commit, and the run goes ahead. The starter templates' apply jobs set both: `YODEL_LOCK_WAIT=10m` and `YODEL_STAND_DOWN=1`. To clear a lock by hand, once you know nothing is applying: on ClickHouse `ALTER TABLE .lock DELETE WHERE name = 'apply'` (or remove the lock file the refusal names); on Postgres end the holding session, `SELECT pg_terminate_backend()`. `yodel status` and `yodel plan` take no lock. ## Applying `yodel apply ` runs the project's migrations Op ([Approval](/sql-yodeler/approval/)). Under the lock, its Apply step checks the plan digest again, checks every applied migration's checksum, then runs the pending migrations in chain order, statement by statement, writing a `started` row before each and a `succeeded` or `failed` row after. From the ClickHouse example: ```console $ npx yodel apply dev Migrations in dev (clickhouse 26.8.15.10 at 127.0.0.1:8123, history yodeler.history, topology single (default)) Applied (1): 20261010T1722-baseline 2026-10-10 17:22:07.370595 by yodel@example Pending (1): 20261010T1722-add-country Plan digest: jcs1-sha256:c6a49076c4a2b58c984b7a73d59e86c5788e82cb89728c5ec24307869e72bd03 Running migrate-dev (/clickhouse/ops/migrate-dev.op.ts) lock: no KeeperMap on this server (KeeperMap is disabled because 'keeper_map_path_prefix' config is not defined. (BAD_ARGUMENTS) (version 26.8.15.10 (official build))); the lock is a file on this machine, so cross-runner locking needs Keeper (and keeper_map_path_prefix) approval: migrate-dev / approve-migrate-dev for jcs1-sha256:c6a49076c4a2b58c984b7a73d59e86c5788e82cb89728c5ec24307869e72bd03, by yodel at 2026-10-10T17:22:18.586Z 20261010T1722-add-country: applying statement 0: ok (SQLCH201 metadata, shop.events) 20261010T1722-add-country: applied [phase] Plan ✓ shellCmd(cmd=yodel apply dev --digest) 1.0s [phase] Approve ✓ gate:approve-migrate-dev() 227ms [approved] yodel at 2026-10-10T17:22:18.586Z [phase] Apply ✓ shellCmd(cmd=yodel apply dev --execute, env={"YODEL_APPROVED_PLAN":"jcs1-sha256:c6a49076c4a2b58c984b7a73d59e86c5788e82cb89728c5ec24307869e72bd03"}) 1.8s Op "migrate-dev" completed in 3.2s Applied: 20261010T1722-add-country. ``` On ClickHouse each statement is rendered for the environment's topology as it is sent ([Topology](/sql-yodeler/topology/)). On Postgres: - A statement that can run in a transaction runs in one of its own, with its `succeeded` row, so the two commit together or not at all. - `CREATE INDEX CONCURRENTLY` and the other statements that cannot run in a transaction run alone, with their row written right after. Before each attempt at a `CONCURRENTLY` build, an INVALID index of that name left by an earlier failed attempt is dropped; a valid one is never touched. - Every statement runs under `lock_timeout` (the profile's `lockTimeoutMs`, default 5 s) and a `statement_timeout` (`statementTimeoutMs`, default 60 s, for a catalog change; `scanTimeoutMs`, default none, for one that reads or rewrites rows). A statement that times out waiting for a lock is tried again after a pause that doubles from 250 ms, up to `YODEL_LOCK_RETRIES` more times (default 5); any other error fails it at once. ## Pre-migration checks Some migrations are only safe when the data looks a certain way: a table is empty before it is dropped, a column has no NULLs before it becomes NOT NULL. A migration says so with `checks` in its `migration.json`, read-only queries that run against the environment before its first statement. Write one with `yodel new`: ```sh npx yodel new drop-legacy --check "legacy-empty: SELECT 1 FROM shop.legacy LIMIT 1" ``` `--check` is repeatable. Written `: ` the check takes that name; otherwise it is `check-1`, `check-2` and so on. In `migration.json` it looks like this: ```json "checks": [ { "name": "legacy-empty", "sql": "SELECT 1 FROM shop.legacy LIMIT 1" } ] ``` What a check must return is its `expect`: | `expect` | passes when the query returns | |---|---| | `"empty"` (the default, and what `--check` writes) | no rows | | `"true"` | one row whose first column is true or 1 | | `{ "min": 1, "max": 1000000 }` (either bound may be left out) | one row whose first column is a number within the bounds, inclusive | A bare table name means the migration's default database (ClickHouse) or schema (Postgres). A check runs read-only: on ClickHouse with `readonly = 2`, on Postgres in a `READ ONLY` transaction that is rolled back, under the profile's `lockTimeoutMs` and `scanTimeoutMs`. `yodel apply ` runs the first pending migration's checks before it runs the migrations Op, so a run that would be refused stops before anyone is asked to approve it. The Op's Apply step runs every pending migration's checks again under the lock, just before that migration's first statement, when earlier migrations in the same run have already applied. A check that fails, or whose query the server refuses, refuses the apply with exit 4 and names the check; nothing of that migration is sent. Under the lock the history records the outcome: the migration's `started` row carries the checks' results in `note`, and a refusal is a `failed` migration row whose `error` names the check. A migration resuming part way does not run its checks again, since they held before its first statement. In the Postgres example a CHECK constraint's validation has a pre-check ([Statements that can fail on the data](/sql-yodeler/lint/#statements-that-can-fail-on-the-data)), and one order has an amount of 0: ```console $ npx yodel apply dev yodel apply: refused: a pre-migration check of 20261010T1723-coupons failed before the gate. Nothing of 20261010T1723-coupons was applied: step 1, ALTER TABLE shop.orders ADD CONSTRAINT orders_amount_positive CHECK (amount > 0) NOT VALID, would fail: 1 rows that fail the check orders_amount_positive (amount > 0) (pre-check precheck-1: SELECT count(*) AS n FROM shop.orders WHERE NOT (amount > 0)) Fix the data, then run yodel apply again; if the check is wrong, edit it in migration.json and run yodel lint --update-checksum 20261010T1723-coupons (the plan digest moves, so the plan is approved again). A statement's pre-check is chant's count of the rows the statement fails on: fix those rows, or change the declaration. [exit 4] ``` The checks are part of `migration.json`, so of the checksum and the plan digest: adding, editing or removing one moves the digest, and the plan is approved again. To change a check of a migration not yet applied anywhere, edit it and run `yodel lint --update-checksum `. `yodel plan` lists each pending migration's checks, and the pull request comment shows them in a table under the migration's steps. ## Resuming a migration that failed part way A statement whose latest row is `succeeded` is never sent again. So when a statement fails, the next `yodel apply` resumes the migration at that statement. (A statement whose latest row is `started`, because the runner stopped while it ran, is sent again: the history cannot tell whether the server ran it.) An Op step that failed resumes from its receipts ([Data migrations](/sql-yodeler/steps/)). In the Postgres example a migration adds a column, then builds an index on it, and someone has made an index of the same name on dev by hand. The part of the run after the gate: ```console $ npx yodel apply dev ... lock: held in Postgres advisory lock (1498367052, 748638931) for history schema yodeler approval: migrate-dev / approve-migrate-dev for jcs1-sha256:b5f00cfef13f9274abb0a096b95bf02bd40883ba91189a20245a3b85e46dedcd, by yodel at 2026-10-10T17:24:17.565Z 20261010T1724-refunds: applying statement 0: ok (SQLPG201 metadata, shop.orders) statement 1: failed: relation "orders_refunded_at_idx" already exists yodel apply: 20261010T1724-refunds failed at statement 1: relation "orders_refunded_at_idx" already exists Fix the cause (on the server, or that statement in the migration's files), then run yodel apply again: it resumes at statement 1 and never sends again the statements that succeeded. ... [exit 1] ``` `yodel status` shows where it stopped, and exits 3 (pending): ```console $ npx yodel status dev Migrations in dev (postgres 18.6 at 127.0.0.1:5432/postgres, history yodeler.history) Applied (2): 20261010T1723-baseline 2026-10-10 17:23:54.742760 by yodel@example 20261010T1723-coupons 2026-10-10 17:24:09.819313 by yodel@example Pending (1): 20261010T1724-refunds (failed part way at statement 1; resumes at statement 1) error: statement 1: relation "orders_refunded_at_idx" already exists Plan digest: jcs1-sha256:f4538cde31ddae8c82df1d6e19ff46886b6e7ebe9ab0d3c22e24451ed8a52c30 [exit 3] ``` The history changed since the approval (statement 0 ran), so the plan digest moved and the gate asks for a new approval. After dropping the hand-made index and approving again, the next run sends only statement 1: ```console $ npx yodel apply dev ... Running migrate-dev (/postgres/ops/migrate-dev.op.ts) lock: held in Postgres advisory lock (1498367052, 748638931) for history schema yodeler approval: migrate-dev / approve-migrate-dev for jcs1-sha256:9527e04f668b47281769059b63977f8fafdbbd3bf06b377854aa05273ad5db09, by yodel at 2026-10-10T17:24:27.531Z 20261010T1724-refunds: resuming at statement 1 statement 0: succeeded in an earlier run; not sent statement 1: ok (SQLPG240 concurrently, shop.orders_refunded_at_idx) statement 2: ok (SQLPG240 concurrently, shop.orders_refunded_at_idx) 20261010T1724-refunds: applied ... ``` ## yodel status `yodel status ` lists the environment's migrations against its history: applied, pending, out of order, failed part way (with the statement it resumes at), and checksum mismatches, and prints the plan digest `yodel apply` would ask approval for. It reads only and takes no lock. Exit codes: 0 every migration applied, 3 migrations pending, 4 a checksum mismatch or an out-of-order migration, 2 a pending migration has a step `yodel apply` does not run, 1 the status could not be read. `--json` prints it as JSON. ```console $ npx yodel status dev Migrations in dev (postgres 18.6 at 127.0.0.1:5432/postgres, history yodeler.history) Applied (7): 20261010T1723-baseline 2026-10-10 17:23:54.742760 by yodel@example 20261010T1723-coupons 2026-10-10 17:24:09.819313 by yodel@example 20261010T1724-refunds 2026-10-10 17:24:31.521713 by yodel@example 20261010T1724-rename-email 2026-10-10 17:24:50.116164 by yodel@example 20261010T1724-drop-coupon 2026-10-10 17:25:09.412058 by yodel@example 20261010T1725-add-gift 2026-10-10 17:25:40.602850 by yodel@example 20261010T1725-index-placed-at 2026-10-10 17:25:24.896277 by yodel@example repaired 2026-10-10 17:25:30.589306 by yodel@example: rebased onto add-gift after it ran on dev Pending: none ``` ## Out-of-order migrations A pending migration that the chain puts before one already applied is out of order. It can happen when a migration that ran in an environment is moved in the chain afterwards, as below. `yodel apply` refuses it (exit 4) and never skips it: ```console $ npx yodel apply dev yodel apply: refused: 20261010T1725-add-gift is out of order: the chain puts it before 20261010T1725-index-placed-at, which is already applied. Nothing was applied. Pass --allow-out-of-order to apply it now (in chain order); it is never skipped. [exit 4] ``` `--allow-out-of-order` applies it, in chain order, behind the same approval as any other run: ```console $ npx yodel apply dev --allow-out-of-order ... Running migrate-dev (/postgres/ops/migrate-dev.op.ts) lock: held in Postgres advisory lock (1498367052, 748638931) for history schema yodeler approval: migrate-dev / approve-migrate-dev for jcs1-sha256:a6b43ee382e0a02028ff147d6d6c1a5e0932f1351c226c64555200835edd5cf9, by yodel at 2026-10-10T17:25:36.714Z 20261010T1725-add-gift: applying (out of order, allowed) statement 0: ok (SQLPG201 metadata, shop.orders) 20261010T1725-add-gift: applied ... ``` ## Forks and yodel rebase Two branches that each add a migration give one parent two children. Merged together, `yodel lint` fails on the fork (an error that cannot be lowered), naming both: ```console $ npx yodel lint migrations/ error fork: 20261010T1725-add-gift and 20261010T1725-index-placed-at both follow 20261010T1724-drop-coupon; rebase one onto the other 20261010T1723-baseline warning SQLPG112: step 2: orders.customer_email looks like a secret or personal data and neither it nor shop.orders has a COMMENT; say what it holds and how it is protected 20261010T1723-coupons warning data-dependent: step 1: SQLPG217 Add a constraint NOT VALID on shop.orders: can fail on the rows already there (rows that fail the check orders_amount_positive (amount > 0)); yodel apply runs this pre-check before the migration's first statement and refuses when it is not 0: SELECT count(*) AS n FROM shop.orders WHERE NOT (amount > 0) 20261010T1724-drop-coupon silenced destructive: step 0: SQLPG204 Drop a column on shop.orders: removes data that cannot be recovered reason: no order ever had a coupon; checked on dev 20261010T1724-rename-email warning SQLPG112: step 0: orders.email looks like a secret or personal data and neither it nor shop.orders has a COMMENT; say what it holds and how it is protected 7 migrations: 1 error, 3 warnings, 1 silenced. [exit 3] ``` `yodel rebase [] [--onto ]` moves a migration onto a new parent: its change (its recorded schema against its old parent's) is applied to the new parent's recorded schema, its statements are diffed again, and its files and checksum are written again. Migrations after it move with it. Ids do not change. With no `` it resolves the directory's one fork by moving the newer of the two (by the timestamp in its id) onto the end of the other's branch. A migration that ran somewhere must not be rebased: its statements already ran in that order. `--env ` (repeatable) reads each environment's history first and refuses to move a migration it records: ```console $ npx yodel rebase 20261010T1725-index-placed-at --onto 20261010T1725-add-gift --env dev yodel rebase: refused: dev's history records 20261010T1725-index-placed-at (applied). Nothing was written. A migration that ran is never rebased: its statements already ran in that order. Rebase the other branch onto it instead (yodel rebase --onto ). [exit 4] ``` The fix it names is to move the other migration instead: here `yodel rebase --onto --env dev`. Without `--env`, no history is checked, and the rebase says so: ```console $ npx yodel rebase 20261010T1725-index-placed-at --onto 20261010T1725-add-gift Rebased 20261010T1725-index-placed-at onto 20261010T1725-add-gift (it followed 20261010T1724-drop-coupon). 20261010T1725-index-placed-at: follows 20261010T1725-add-gift; 2 statements; checksum sha256:8b0a50449f82a2bc1c63fb8c868eb6968dba13b9ab5b02ccc43c8fd231659117 -> sha256:6be6d45f55c4fa14ed67b2b22471f217f92ba3115bcc5c10fa154c99ef1a281f No history was checked (--env): make sure none of these ran anywhere before merging. ``` `yodel rebase` also refuses (exit 4, nothing written) when both branches change the same object differently (resolve it by hand: delete the migration and run `yodel new` again), when a migration it would rewrite was edited by hand (unless `--force`, which drops the edits), or when the moved migration adds nothing on its new parent. A `-- yodel:allow` line ([Lint](/sql-yodeler/lint/)) does not count as an edit by hand: the rebase writes it again above the same statement, and says how many it carried over. When the statement a line silenced is gone from the rebased migration or changed, the rebase refuses (exit 4, nothing written) and names the line; remove it, record the checksum with `yodel lint --update-checksum `, rebase, and silence what the new statements need. ## Checkpoints: yodel checkpoint A long chain makes every new environment, and every replay, run every migration from the first. `yodel checkpoint ` writes a migration after the newest one whose statements create the newest migration's recorded schema from nothing (what `yodel new` writes for a first migration). Its recorded schema is its parent's, and its `migration.json` marks it a checkpoint with the SHA-256 of that schema: `"checkpoint": { "schema": "sha256:..." }`. It reads no database and does not build `src/`. Its `migration.sql` starts `-- yodel checkpoint `. How `yodel apply` and `yodel status` treat one: - An environment whose history is empty starts at the newest checkpoint. The migrations before it are skipped there, never run; after the checkpoint, the chain goes on as usual. - An environment whose history records a checkpoint started there, and goes on skipping the migrations before it. - Any other environment, one that ran the migrations a checkpoint stands for, skips the checkpoint. A skipped migration is neither applied nor pending, so it is never out of order. `yodel status` lists them under "Skipped" (`skipped` in `--json`, each `covered` by the checkpoint it names, or a `checkpoint` the environment does not start from). The checkpoint is a migration like any other, so its checksum covers its statements, its mark and its recorded schema. `yodel lint` adds two checks: - rule `checkpoint` (an error, offline): its recorded schema still hashes to its mark, and so does its parent's. An edit to either, even with the checksum recorded again, fails here. - `--replay ` replays the chain without the checkpoint, then the checkpoint alone into the emptied database, and compares the result with its recorded schema. Statements that do not create that schema fail the replay (rule `replay`), naming the checkpoint. With both, a fresh environment that starts from the checkpoint ends at the schema an environment that ran the chain has. Write another checkpoint later the same way; a fresh environment starts at the newest. `yodel checkpoint` refuses (exit 4) when the newest migration is a checkpoint already. ## Reverting: yodel revert `yodel revert ` undoes migration `` in ``. Each migration records the schema after it, and its parent records the schema before it, so the reverse is planned offline: the statements that take ``'s recorded schema back to its parent's (an empty schema for the first migration), diffed by chant as `yodel new` diffs. Only the newest migration applied in the environment can be reverted, since a later one was written against it; anything else refuses (exit 4). So does a checkpoint the environment started from (it created the whole schema, and nothing before it ran there), and a migration a checkpoint left out there ([Checkpoints](#checkpoints-yodel-checkpoint)); both refusals name the checkpoint. `--dry-run` prints the statements, each with its rule and class (a drop is marked destructive), and the digest, and runs nothing. It goes through the environment's migrations Op, as `yodel apply` does. The Op's Plan step prints the revert's digest (its statements, the history and the live schema), the gate binds the approval to that digest, and the Apply step plans the revert again under the lock, refuses if the digest moved, checks the approval in chant's gate ledger, and runs it. A run that stops at the gate prints the command that approves the revert, `yodel approve --plan ` (exit 3); approve, then run `yodel revert` again. The policy in `yodel.config.ts`, read at the base commit, is judged over the revert's statements before the gate and again under the lock, as it is over an apply: a rule that refuses drops refuses a revert that drops what the migration added (exit 4), unless `yodel override --revert ` recorded an override for the revert's digest ([Policy and overrides](/sql-yodeler/approval/#reverts)). Statements carry the schema back, never data. A migration with a data step (a backfill) is refused (exit 2) unless `--step ` names a hand-written revert step: SQL statements, each ending with `;` at the end of a line, that undo what the data step did. They run before the reverse statements, and their text is in the digest. A reverse that would need an Op or a manual step (a ClickHouse sort key changed back, a Postgres column renamed back) is refused (exit 2): write it as a new migration with `yodel new`. The history gets a `revert` row for each statement as it starts, succeeds or fails, then a migration row whose status is `reverted`. A revert that fails part way resumes at the statement that failed, and the migration stays applied until the `reverted` row is written. After it, the migration is pending again, and `yodel status` marks it `reverted by ` (`reverted` in `--json`); the audit log of `yodel report` lists the revert. The next `yodel apply` would apply it again, every statement of it, so delete it or rewrite it (`yodel new --replace`) first. On Postgres, a history table made before `yodel revert` existed has its `kind` and `status` checks widened the first time a revert writes to it. ## yodel repair An applied migration whose files changed no longer matches the checksum its history recorded, and `yodel status` and `yodel apply` refuse: ```console $ npx yodel apply dev yodel apply: refused: the history does not match the migrations directory; nothing was applied: 20261010T1725-index-placed-at was applied with checksum sha256:8b0a50449f82a2bc1c63fb8c868eb6968dba13b9ab5b02ccc43c8fd231659117, but its files now hash to sha256:6be6d45f55c4fa14ed67b2b22471f217f92ba3115bcc5c10fa154c99ef1a281f; if the edit is intended, record it with yodel repair 20261010T1725-index-placed-at --reason ... [exit 4] ``` When the edit is intended (here, the rebase above; or a statement edited after a server upgrade made it invalid), `yodel repair --reason "" ` appends a row with the files' new checksum and the reason, under the apply lock. Nothing runs on the server: the edit is recorded, not applied. Repair each environment the migration ran in. ```console $ npx yodel repair 20261010T1725-index-placed-at --reason 'rebased onto add-gift after it ran on dev' dev Repaired 20261010T1725-index-placed-at in dev's history (yodeler). old checksum: sha256:8b0a50449f82a2bc1c63fb8c868eb6968dba13b9ab5b02ccc43c8fd231659117 new checksum: sha256:6be6d45f55c4fa14ed67b2b22471f217f92ba3115bcc5c10fa154c99ef1a281f reason: rebased onto add-gift after it ran on dev ``` `yodel status` then lists the migration with the time it was applied and who applied it, and the repair (when, by whom, and why) under it, as in [yodel status](#yodel-status) below. It refuses (exit 4) when the migration is not applied in that environment, when its files already match, or when it is gone from the directory. When the edit is to a migration applied nowhere yet, there is no history to repair: `yodel lint --update-checksum ` writes the new checksum into its `migration.json` ([Lint](/sql-yodeler/lint/)). ## Proven by The scenario claims below run what this page describes against a real server, once plain (it passes) and once with the behaviour broken (the claim catches it). [Claims status](/sql-yodeler/claims/) lists every claim. | Claim | What it says | Plain, broken | Last run | |---|---|---|---| | `new` | yodel new writes the next migration offline, with no dev database, and it applies; on Postgres, functions, procedures and triggers too, which lint checks | ClickHouse: pass, caught; Postgres: pass, caught | `c6f58a4`, 2026-10-10 | | `resume` | an interrupted apply resumes where it stopped: a backfill step written as yodel's form (table, key, batch size, SQL) from its receipts, running each batch once, and a failed statement at that statement, never resending one that ran | ClickHouse: pass, caught; Postgres: pass, caught | `c6f58a4`, 2026-10-10 | | `out-of-order` | a migration merged late, before one already applied, is refused unless --allow-out-of-order, and never skipped | ClickHouse: pass, caught; Postgres: pass, caught | `c6f58a4`, 2026-10-10 | | `lock-retry` | Postgres: a statement blocked behind another session's lock times out at lock_timeout and is retried until it succeeds; Postgres and ClickHouse with Keeper: two applies at once never send a statement twice, the second waits naming the holder, and --stand-down steps aside for a newer commit | ClickHouse: pass, caught; Postgres: pass, caught | `c6f58a4`, 2026-10-10 | | `approval` | a pending migration applies only after chant approve of its plan digest, and the history records that digest; a failing pre-migration check refuses it, and so does a policy rule, read at the base commit, unless an override is recorded for that digest, for a yodel revert as for an apply; the audit log derived from the history and the ledger accounts for every approval and apply, and names one removed; a reader and a writer that are one user are refused, and the reader, its password a minted token, cannot write; an approval is refused once the migration changed after it, and nothing is applied | ClickHouse: pass, caught; Postgres: pass, caught | `c6f58a4`, 2026-10-10 | | `lint` | yodel lint --replay replays the migrations into a fresh database and each gives the schema it recorded; offline, yodel lint fails a migrations directory with a fork (exit 3), naming both migrations; a checkpoint replays alone to its recorded schema, and a fresh environment starts from it; yodel revert undoes the newest migration behind the gate, with a hand-written step for its data step, back to its parent's recorded schema, and refuses a checkpoint or a migration before one; yodel test runs a project's tests on databases replayed from the migrations, a seed meeting the backfill after it, fails a case whose assertion does not hold, and refuses an environment with a history; on Postgres, a unique index over rows with duplicates is refused before the gate by its generated pre-check (exit 4, naming the statement and the count), and yodel lint flags it as data-dependent | ClickHouse: pass, caught; Postgres: pass, caught | `c6f58a4`, 2026-10-10 | --- # Access control Source: https://intentius.io/sql-yodeler/access/ ## Optional: hand this page to your coding agent ```text Declare the access I describe in src/, next to the tables it is about, following https://intentius.io/sql-yodeler/access/: row-level security, policies, roles and grants on Postgres; users, roles, row policies and grants on ClickHouse. Keep to what the page says the schema owns. Set `access: true` in yodel.config.ts for the environments I name, run `npx yodel new ` and `npx yodel lint`, and open a pull request with src/, the config and the new migration. Never run `yodel apply` against a shared environment, never run `chant approve`, `yodel approve` or `yodel override`, never edit the `chant/lifecycle` branch or `.chant/allowed_signers`, never merge; approvals and applies belong to people. ``` Who can read and write what is part of the schema. chant's sql lexicon declares it next to the tables it is about. On Postgres that is row-level security on a table, policies, the roles the schema needs, grants and revokes, and default privileges. On ClickHouse it is users, roles, row policies and grants ([ClickHouse](#clickhouse) below). yodel plans and applies those declarations like the rest of the schema, shows them under review with their rules and classes, and adopts the ones a database already has. ```ts import { grant, policy, role, table } from "@intentius/chant-lexicon-sql/postgres"; import { app } from "./schema.js"; export const reader = role`CREATE ROLE app_reader NOLOGIN`; export const tickets = table` CREATE TABLE ${app}.tickets ( id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY, tenant text NOT NULL, title text NOT NULL ); ALTER TABLE ${app}.tickets ENABLE ROW LEVEL SECURITY; ALTER TABLE ${app}.tickets FORCE ROW LEVEL SECURITY`; export const ticketsByTenant = policy` CREATE POLICY tickets_tenant ON ${tickets} FOR SELECT TO ${reader} USING (${tickets.columns.tenant} = current_setting('app.tenant'))`; export const useApp = grant`GRANT USAGE ON SCHEMA ${app} TO ${reader}`; export const readTickets = grant`GRANT SELECT ON ${tickets} TO ${reader}`; export const editTitles = grant`GRANT UPDATE (title) ON ${tickets} TO support`; ``` chant's page on access (`postgres-access` in the sql lexicon's docs) has the declarations' full syntax and how privileges are compared. ## Turn it on per environment Access is off until an environment turns it on, on either dialect, so a project whose access is managed somewhere else sees no change. Set it in `yodel.config.ts`: ```ts export default defineConfig({ environments: { dev: { access: true }, prod: { access: true }, }, }); ``` The first one set wins: `environments..access` in `yodel.config.ts`, then chant's `sql.profiles..access` in `chant.config.ts`, then off. yodel hands the result to chant for everything it runs itself: `yodel plan`, `yodel new`'s plan, `yodel init --from` and the server reads of the versioned path. On the declarative path, the apply is the project's `ApplyOp` run by `chant run`, and `yodel drift` runs `chant lifecycle diff`. Both read `chant.config.ts` themselves, so there `yodel.config.ts` must agree with the profile. yodel refuses otherwise rather than plan one thing and apply another: ```text yodel plan: yodel.config.ts sets environments.dev.access to true, but the declarative path applies through chant, which reads sql.profiles.dev.access in chant.config.ts (unset, so off). Set sql.profiles.dev.access to true as well, or leave access to chant.config.ts. ``` On Postgres, with access off, the access declarations are left out of the plan (chant's hint counts them), the server's policies and privileges are not read, and a table is created without its `ROW LEVEL SECURITY` statements. With access on, the declared privileges are all the privileges on the declared objects: one granted by hand is revoked by the next apply. The same holds on ClickHouse: with access off, chant leaves the users, roles, row policies and grants out of the plan and the apply and does not read the server's, and `yodel drift` lists each access declaration as not compared. ## What the schema owns and what the environment owns This section and the next three are about Postgres; ClickHouse has [its own](#clickhouse). The schema owns these, declared and kept as declared: - row-level security on a table, and `FORCE ROW LEVEL SECURITY`; - policies; - privileges on the declared schemas, tables, views, sequences, columns, functions and procedures; - default privileges of the role that applies (and of a role named with `FOR ROLE`); - the roles the schema itself needs, such as a `NOLOGIN` role that carries privileges. They are created when missing and are never dropped. The environment owns these, and they are never declared: - Passwords. `PASSWORD` in `CREATE ROLE` is refused. Set a role's password wherever the environment keeps its credentials; a migration file, a plan comment or the history never holds one. - Memberships: who is in which role. `IN ROLE`, `ROLE`, `ADMIN` and `GRANT role TO role` are refused. They differ per environment, and the platform usually provisions login roles and their memberships. - Roles the environment provisions. A declaration names one as text where it grants to it (`TO support`); yodel never creates, reads or drops it. A grant to a role the server does not have fails in the apply, naming it. - Who owns an object. ## Under review `yodel plan` on the declarative path ends with an access section: every access change with its rule and class, and whether access is managed in the environment and where that is set. A revoke that no `REVOKE` declaration asks for takes away a privilege granted by hand (or one whose declaration was deleted). It changes what a role can do once the apply runs, so the plan lists it on its own: ```text Access control (managed in dev, from yodel.config.ts): 1 change SQLPG297 metadata relation app.tickets TO app_reader privileges: insert, select -> select (granted by hand) Revoked by this apply, though no declaration revokes it: privileges granted by hand (or whose declaration was deleted). The role loses it once the apply runs: relation app.tickets FROM app_reader: insert To keep it, declare it with grant`...`. ``` `yodel plan --json` has the same under `access`: `managed`, `source`, and `changes`, a revoke of a hand-made grant marked `byHand: true`. The apply prints the section before it runs. Its gate binds the digest of chant's `lifecycle diff`, which observes policies and roles but not privileges, so an approval on the declarative path does not cover a privilege granted or revoked by hand after it: the apply makes the privileges match the declarations whatever changed in between. On the versioned path every statement, grants and revokes included, is in the migration's checksum and so in the digest the approval binds. On the versioned path, `yodel new` writes each access change as a statement with its rule and class (SQLPG200 for a role or policy created with its table, SQLPG290 to SQLPG298 otherwise), and prints how many steps are about access and which of them take access away. `yodel plan` and the pull request comment list the same under each pending migration. | Rule | Change | Class | |---|---|---| | SQLPG290 | A policy created on an existing table | metadata | | SQLPG291 | A policy changed | metadata | | SQLPG292 | A policy dropped | drop | | SQLPG293 | Row-level security turned on or off, or forced | metadata | | SQLPG294 | A role's attributes changed | metadata | | SQLPG296 | A privilege granted | metadata | | SQLPG297 | A privilege revoked | metadata | | SQLPG298 | Default privileges changed | metadata | ## Lint and the gate Two lint rules read access steps ([Lint](/sql-yodeler/lint/)): - `pg-access` (an error, silenceable): a statement that takes access away. That is a revoke, a dropped policy, or row-level security turned off or no longer forced. A revoke on an object the same migration creates, such as `REVOKE EXECUTE ... FROM PUBLIC` on a new function, takes nothing from anyone and is not flagged. - `pg-rls-not-forced` (a warning, silenceable): a migration that enables a table's row-level security while its recorded schema does not force it. Postgres does not hold a table's owner to its policies unless the table forces them, and the owner is usually the role that applies migrations. A wave with gate `on-destructive` waits on a `pg-access` step as it waits on a drop ([Approval](/sql-yodeler/approval/#waves-a-gate-per-environment)). ## Adopting a database With access on for the environment, `yodel init --from ` adopts each table's row-level security with the table, the policies, and the privileges as `grant` declarations. Each declaration is the `GRANT` or `REVOKE` that takes a new object's privileges to what the server holds, so the adoption plans as no change. Roles are the environment's and are not adopted; the grants name them as text. The baseline migration records the policies and grants with the rest, and nothing is sent to the server. ## ClickHouse On ClickHouse the access declarations are `user`, `role`, `policy` (a row policy) and `grant`, from `@intentius/chant-lexicon-sql/clickhouse`: ```ts import { grant, policy, role, user } from "@intentius/chant-lexicon-sql/clickhouse"; import { events } from "./schema.js"; export const reader = role`CREATE ROLE app_reader SETTINGS max_threads = 4`; export const dashboards = user`CREATE USER dashboards DEFAULT ROLE ${reader}`; export const clicksOnly = policy`CREATE ROW POLICY clicks ON ${events} FOR SELECT USING kind = 'click' TO ${reader}`; export const readEvents = grant`GRANT SELECT ON ${events} TO ${reader}`; export const dashboardsRead = grant`GRANT ${reader} TO ${dashboards}`; ``` chant's page on ClickHouse access (`clickhouse-access` in the sql lexicon's docs) has the full syntax and how each is compared. What the schema owns and what the environment owns: - A password is the environment's. `IDENTIFIED BY` is refused in a declaration. A user declared without `IDENTIFIED`, like `dashboards` above, is not created by chant: the apply reports it as not attempted and its grants as waiting on it until the environment creates the user with its password; from then on the apply keeps the rest of it (its default role, host, settings) as declared and leaves the password alone. A user declared with a method that holds no secret (`NOT IDENTIFIED`, `ssl_certificate`, `ldap`, `kerberos`, `http`, `ssh_key`) is created by chant. - Grants are compared per grantee. All the `grant` declarations naming a grantee are everything it holds: anything else it holds is revoked by the next apply, and there is no `REVOKE` declaration. - Users, roles and row policies are created and changed, never dropped. None of them carries chant's ownership marker, so a plan reads only the ones the build declares, and one made by hand is never seen. ### Under review `yodel plan` and `yodel apply` on the declarative path end with the same access section as on Postgres. Every revoke on ClickHouse takes away something no declaration grants, so each one is called out: ```text Access control (managed in dev, from yodel.config.ts): 1 change SQLCH274 metadata grants app_reader grants.INSERT ON shop.events: INSERT ON shop.events -> nothing (granted by hand) Revoked by this apply, though no declaration revokes it: privileges granted by hand (or whose declaration was deleted). The role loses it once the apply runs: INSERT ON shop.events FROM app_reader To keep it, declare it with grant`...`. ``` `yodel drift` reports the same grant: chant's `lifecycle diff` reads users, roles and row policies but leaves grants unobserved, so yodel compares each declared grantee's grants itself and lists a grant held by hand, a declared grant that is missing, and a grantee that is gone. On the versioned path `yodel new` writes the access statements with their rules and classes, prints how many steps are about access and which take it away, and `yodel plan` and the pull request comment list the same under each pending migration. | Rule | Change | Class | |---|---|---| | SQLCH200 | A user, role or row policy created, or a grantee's first grants | create | | SQLCH270 | A role's settings changed (`ALTER ROLE`) | metadata | | SQLCH271 | A user's clause changed (one `ALTER USER` per clause) | metadata | | SQLCH272 | A row policy changed (`CREATE ROW POLICY OR REPLACE`) | metadata | | SQLCH273 | A privilege or role granted | metadata | | SQLCH274 | A privilege or role revoked | metadata | `ch-access` (an error, silenceable) flags a revoke, and a wave with gate `on-destructive` waits on it ([Lint](/sql-yodeler/lint/)). ### Topology and replay None of these objects belongs to a database. In the `cluster` topology, and in `replicated` when it names a cluster, their statements are sent `ON CLUSTER` (`CREATE USER ... ON CLUSTER`, `GRANT ON CLUSTER ...`); inside a `Replicated` database alone they are sent to the node, since its log does not carry them ([Topology](/sql-yodeler/topology/)). `yodel lint --replay` empties the databases it replays into, and also drops the functions, users, roles and row policies the history declares, which outlive a database. A user the history declares without `IDENTIFIED` is the environment's to create, so the replay creates it with a random password before the first statement, and gives it its declared clauses before each migration is compared. ### Adopting With access on for the environment, `yodel init --from ` writes `src/access.ts` beside the import's `schema.ts`: - every row policy on a table in the profile's `databases`; - each user or role holding a privilege in those databases, each role a policy applies to, and each user or role those roles are granted to, but not the server's `default` user; - for each of them, its `CREATE USER` (without its authentication) or `CREATE ROLE`, and all of its grants as `grant` declarations, not only those in the databases, since a grantee's grants are compared as a whole. A grantee with a partial revoke (`REVOKE` in `SHOW GRANTS`) cannot be declared and is left out with a warning. The baseline records the grants with the rest, and nothing is sent to the server. With access off, init adopts the databases, tables, views, dictionaries and functions alone. ## Proven by The scenario claims below run what this page describes against a real server, once plain (it passes) and once with the behaviour broken (the claim catches it). [Claims status](/sql-yodeler/claims/) lists every claim. | Claim | What it says | Plain, broken | Last run | |---|---|---|---| | `declarative` | the declarative path: yodel plan shows the change against the live server and yodel apply makes it behind the plan-bound gate, for src/ declarations (on Postgres, functions, procedures and triggers, and access control: a role, a policy and grants, with a grant made by hand revoked; on ClickHouse, a dictionary, a function, and access control: a role, a user, a row policy and grants, with a grant made by hand revoked) and for an ORM's exported DDL | ClickHouse: pass, caught; Postgres: pass, caught | `c6f58a4`, 2026-10-10 | | `adopt` | yodel init --from adopts a live database without touching it, and yodel plan then shows no change; on Postgres its policies, row-level security and grants too, on ClickHouse its dictionaries, functions, roles, users, row policies and grants; yodel init --baseline records the baseline, behind the gate, in a second environment that holds the same schema, and refuses one that differs | ClickHouse: pass, caught; Postgres: pass, caught | `9329873`, 2026-10-10 | --- # Approval Source: https://intentius.io/sql-yodeler/approval/ ## Optional: hand this page to your coding agent ```text An apply was refused or is waiting for approval. Following https://intentius.io/sql-yodeler/approval/, read the job log and `npx yodel plan ` with the reader's credentials, and tell me what the plan would run, what moved since the last approval, and the approve command. Never run `yodel apply` against a shared environment, never run `chant approve`, `yodel approve` or `yodel override`, never edit the `chant/lifecycle` branch or `.chant/allowed_signers`, never merge; approvals and applies belong to people. ``` `yodel apply` applies only a plan someone approved, on both paths. The approval is chant's (the [Glossary](/sql-yodeler/glossary/) has the terms): a gate in an Op, bound to a plan digest, resolved by a person with `yodel approve` (chant's `approve` underneath), and kept on the `chant/lifecycle` branch of the project's git repository. yodel keeps no approval store of its own. The output quoted here is from the examples' runs; `...` marks lines left out. The runs were recorded before yodel printed `yodel approve` in its messages: where they show `chant approve ...`, yodel now prints `npx yodel approve --plan `, which records the same approval, and the runs approve with chant's command because they had no terminal to ask. ## The migrations Op On the versioned path, `yodel apply ` runs the project's migrations Op for the environment with `chant run`. It is an ordinary chant Op with three phases, which the starter templates and the examples declare in `ops/migrate-.op.ts`: ```ts import { Op, phase, gate, shell, stepOutput } from "@intentius/chant/op"; // yodel apply dev runs this Op when migrations are pending. The gate is bound // to the plan digest (the pending migrations' checksums, the topology, the // history and the live schema), so an approval given for one state never // applies another. const plan = shell("yodel apply dev --digest", { id: "plan" }); const digest = stepOutput(plan, "stdout"); export const op = Op({ name: "migrate-dev", overview: "Apply the pending migrations to dev, behind an approval bound to their plan digest", phases: [ phase("Plan", [plan]), phase("Approve", [gate("approve-migrate-dev", { plan: digest })]), phase("Apply", [shell("yodel apply dev --execute", { env: { YODEL_APPROVED_PLAN: digest }, timeout: "2h" })]), ], }); ``` - Plan: `yodel apply dev --digest` prints the plan digest. - Approve: a gate bound to that output. A resolution counts only for that digest. - Apply: `yodel apply dev --execute`, given the approved digest in `YODEL_APPROVED_PLAN`, takes the lock, computes the digest again, refuses if it moved, and applies. The digest is checked twice: by the gate on every run, against the approval, and by the Apply step under the lock, against the digest the gate passed, so a change between the two (another runner applying first, a table altered out of band) refuses too. `yodel apply` finds the Op by its shape, whatever its name. When there is none it prints this declaration to add; when more than one applies the environment, `--op ` picks one; an Op whose gate is not bound to the Plan step's output is refused. `chant run migrate-dev` run directly (as CI may) needs `yodel` on `PATH`; `yodel apply` puts itself there for the Op it runs. The declarative path's `ApplyOp` carries its own plan-bound gate ([The two workflows](/sql-yodeler/workflows/)). ## The plan digest On the versioned path the digest covers what the run would do and the state it would start from: - the migrations it would apply, in order, each with its files' checksum and the statements of it that already succeeded (a migration that failed part way resumes); - the history it starts from: every applied migration and its checksum; - the topology the statements are rendered for; - the live schema: a fingerprint of the server's catalog in the environment's databases or schemas. The history's own database is left out, so recording a run does not move the digest. It is chant's plan digest (`jcs1-sha256:...`), the one the gate binds. `yodel plan ` prints it with the command that approves it, `npx yodel approve --plan `: ```console $ npx yodel plan dev Plan for dev (clickhouse 26.8.15.10 at 127.0.0.1:8123, history yodeler.history (single (default))): 1 applied, 1 pending 20261010T1722-add-country 0 statement SQLCH201 metadata shop.events ALTER TABLE `shop`.`events` ADD COLUMN country LowCardinality(String) DEFAULT '' AFTER `at` Plan digest: jcs1-sha256:c6a49076c4a2b58c984b7a73d59e86c5788e82cb89728c5ec24307869e72bd03 Approve it with: chant approve migrate-dev approve-migrate-dev --plan jcs1-sha256:c6a49076c4a2b58c984b7a73d59e86c5788e82cb89728c5ec24307869e72bd03 or, at a terminal, with npx yodel approve dev, which shows the plan and asks you first. Not approved yet. Approving binds exactly this digest; if anything it covers changes before the apply runs, the gate asks again and nothing is applied. ``` `yodel status ` prints it too, and `yodel apply --plan` prints the pending migrations and the digest without applying. ## Approving Run the command the plan printed, at a terminal: `npx yodel approve dev --plan ` shows the plan, asks you to type `dev`, and records the approval ([Approve and resume](#approve-and-resume)). The run below, with no terminal to ask, recorded the same approval with chant's command: ```console $ npx chant approve migrate-dev approve-migrate-dev --plan jcs1-sha256:c6a49076c4a2b58c984b7a73d59e86c5788e82cb89728c5ec24307869e72bd03 Gate "approve-migrate-dev" on "migrate-dev" resolved by yodel at 2026-10-10T17:22:18.586Z This approves the plan jcs1-sha256:c6a49076c4a2b58c984b7a73d59e86c5788e82cb89728c5ec24307869e72bd03, and only that plan. A run whose fresh plan differs refuses rather than applying it. This records the resolution as a fact; it does not itself re-run anything. Run `chant run migrate-dev` and it walks through gate "approve-migrate-dev". ``` The approval is a commit on the local `chant/lifecycle` branch. chant pushes it to the remote's `chant/lifecycle` when the repository has a remote, so CI sees it: every job in the starter templates fetches that branch first. In the templates, the pull request comment ends with the approve command; approve before merging, and the apply job after the merge goes through. A merge with nothing approved applies nothing and fails with the command to run; [`yodel approve`](#approve-and-resume) records the approval and starts that job again. Then `yodel apply` runs through the gate: ```console $ npx yodel apply dev ... Running migrate-dev (/clickhouse/ops/migrate-dev.op.ts) lock: no KeeperMap on this server (KeeperMap is disabled because 'keeper_map_path_prefix' config is not defined. (BAD_ARGUMENTS) (version 26.8.15.10 (official build))); the lock is a file on this machine, so cross-runner locking needs Keeper (and keeper_map_path_prefix) approval: migrate-dev / approve-migrate-dev for jcs1-sha256:c6a49076c4a2b58c984b7a73d59e86c5788e82cb89728c5ec24307869e72bd03, by yodel at 2026-10-10T17:22:18.586Z 20261010T1722-add-country: applying statement 0: ok (SQLCH201 metadata, shop.events) 20261010T1722-add-country: applied ... ``` ### Approve and resume `yodel approve ` approves at the terminal without the Op and gate names or the digest. It shows the pending migrations (each statement's class, and any destructive step) and the plan digest, asks you to type the environment's name, and records the same approval `chant approve` gives, bound to the same digest. Then it starts the CI job that waits at the gate again, so the apply goes on without a new push: ```sh npx yodel approve prod ``` That one command is the loop: the waiting job printed the gate and the digest, `yodel approve` records the approval, chant pushes `chant/lifecycle`, and calls the forge to start the job again. It runs `chant approve --plan --resume`, and chant finds the job from the pending record it wrote at the gate: | Forge | What chant calls | The token it needs, at least | |---|---|---| | GitHub | re-runs the run's failed jobs (`POST /repos/{repo}/actions/runs/{run}/rerun-failed-jobs`), at the same commit | a fine-grained token with Actions: read and write on the repository | | GitLab | retries the job (`POST /projects/{id}/jobs/{job}/retry`); the waves after it run when it passes | a project access token with the `api` scope and the Developer role (`CI_JOB_TOKEN` cannot retry a job) | | Forgejo | dispatches `yodel-apply.yml` again on the branch (Forgejo has no re-run API), so the waves before it plan again and apply nothing new | a token with the `write:repository` scope | The token comes from `CHANT_FORGE_TOKEN`, else `GITHUB_TOKEN` or `GH_TOKEN` (GitHub and Forgejo), or `GITLAB_TOKEN` (GitLab). The push of `chant/lifecycle` needs your usual git access to the repository. - `--plan ` names the digest you mean to approve, the one in the pull request comment or the waiting job's log. When it is not the plan that waits, nothing is approved, and yodel names the digest that waits. - `--no-resume` records the approval and starts nothing. Push `chant/lifecycle` and run the job again yourself, or leave it to the [scheduled resume job](#the-scheduled-resume-job). - When the approval is recorded but no job was started (no token, a token the forge refused, a push that did not land), chant says why and `yodel approve` exits 3. Once that is fixed, run the job again, or leave it to the [scheduled resume job](#the-scheduled-resume-job). - When no CI job waits (the gate was reached by `yodel apply` on your machine), there is nothing to resume; run `yodel apply `. In a repository with no remote, `yodel approve` records the approval and resumes nothing. It never approves without a person at the terminal: with no terminal to ask (a script, CI, a coding agent) it approves nothing and exits 4, and no flag skips the question. A script that records approvals itself takes chant's command from the `--json` output's `chantApprove` field. ### The scheduled resume job An approval recorded with `--no-resume`, by a script with `chantApprove`, or one whose resume failed, leaves the job waiting. `yodel.config.ts` `ci.resume` adds a job that starts it on a schedule: ```ts ci: { forges: ["github", "gitlab", "forgejo"], resume: { schedule: "*/10 * * * *" } }, ``` `npm run ci` then renders `yodel-apply-resume`, which runs `chant run resume --op yodel-apply`: every apply wave that waits at its gate and whose approval has arrived since is started again, the way `yodel approve` starts it. It approves nothing, and it skips a run that is still going, passed, or was already started again, so it can run every few minutes. - GitHub: `.github/workflows/yodel-apply-resume.yml`, with the job's own token and `actions: write`. - Forgejo: `.forgejo/workflows/yodel-apply-resume.yml`, with the `CHANT_FORGE_TOKEN` secret, a token with `write:repository` (the job's own token cannot dispatch a workflow). - GitLab: a `yodel-apply-resume` job that runs in scheduled pipelines only. Create a pipeline schedule with the cron (Settings > CI/CD > Schedules); `GITLAB_TOKEN` needs the `api` scope and the Developer role. ## Waves: a gate per environment In SQL Yodeler a wave is one environment (for a tenant set, one environment over many databases or schemas); terragucci uses the word for a batch of Terraform roots applied behind one approval. The starter templates' apply pipeline is a list of waves, one per environment, in the order `yodel.config.ts` gives them. Each wave has its own gate policy: ```ts waves: [ { env: "dev", gate: "never" }, { env: "staging", gate: "on-destructive" }, { env: "prod", gate: "always" }, ], ``` | `gate` | The wave waits for an approval when | |---|---| | `always` (the default) | it has a migration to apply | | `on-destructive` | a pending step drops something, is a Postgres rewrite or lock-heavy statement (lint's `destructive`, `pg-rewrite`, `pg-lock`), a ClickHouse mutation or rebuild (`ch-mutation`, `ch-rebuild`), takes access away (`pg-access`, `ch-access`), or is an Op or manual step | | `never` | never; review and branch protection are all that stand in front of it | terragucci's `on-destroy` is the same rule; SQL Yodeler and chant spell it `on-destructive`, and SQL Yodeler refuses `on-destroy` ([terragucci's gate policies](https://intentius.io/terragucci/reference/stages/#gate-policy)). `npm run ci` renders the waves as `yodel-waves.json`, a chant Op waves spec, and the `yodel-apply` pipeline from it on GitHub, GitLab and Forgejo: one job per wave, each needing the one before. Every job runs `chant run wave`, which: 1. plans the environment with `yodel apply --wave-plan `: the migrations' plan digest, the gate policy, and whether a pending step is destructive, all in one digest; 2. reads the wave's gate policy from `yodel-waves.json` as the commit before the push has it, and decides. A wave that needs an approval applies nothing, prints the command that approves it, `npx yodel approve --plan `, and exits 3, so the waves after it do not run; 3. applies with `yodel apply --wave-apply `, which takes the lock, plans again, and applies only when the plan is still the one the wave decided on and the decision holds: an approval of the wave's digest in the gate ledger, or a policy at the base commit that needs none. To let a waiting wave through, run `yodel approve ` at a terminal: it approves the wave and starts its job again ([Approve and resume](#approve-and-resume)). With `--no-resume`, push `chant/lifecycle` and re-run the failed job (on GitLab, retry it). In a project whose `yodel.config.ts` `waves` names the environment, `yodel approve` approves the wave's gate, `yodel-apply-wave-`, for the wave's set digest, which is what the job waits on, not the migrations Op's gate. It plans the wave as the job does (the plan, the gate policy at the base commit, what the environments before it applied), shows it, and asks you to type the environment's name. A wave whose policy needs no approval for the plan has nothing to approve, and one an earlier environment has not caught up with is refused until it has. The policy comes from the base commit, the first parent of the commit being applied, never from the change. A pull request that sets prod's gate to `never` merges, and prod still waits under the gate main had. Only the gate is read there: the waves, their order and the migrations come from the commit being applied. yodel reads its own config at the base the same way: `YODEL_BASE` names another commit, and on a pull request it is the merge base with the target branch. ### Promotion: what the wave before has applied A wave applies only migrations the environment before it has applied. `waves[].requires` names that environment; it defaults to the wave before, and `requires: false` turns it off. Before anything is asked or applied, `yodel apply ` (in a wave job, run by hand, or the migrations Op's Apply step) reads the required environment's history with that environment's own profile, which outside its own wave job is its reader, and refuses any pending migration the history does not record as applied with the checksum its files have now. It exits 4 and applies nothing, with a message like this one: ```text yodel apply: refused: prod applies only migrations staging applied first, and 1 pending one has not run there: 20261010T0644-add-promo: staging's history does not record it applied (required by yodel.config.ts and yodel.config.ts at aa556870d420) Nothing was applied. Apply it to staging first. ``` The requirement is read from the working tree's `yodel.config.ts` and from the base commit's, and both hold, so a change that drops its own requirement is still held to the base's. A re-run of prod's job, a manual `yodel apply prod`, or a staging wave someone skipped all stop here. The wave plan's digest covers the requirement and which pending migrations the required history records, so prod's approval is of a plan staging has already run: approve prod's wave after staging applied. `yodel plan ` and the pull request comment list, for each pending migration, whether the required environment has run it. The templates give each plan job and each wave job the required environment's reader for that. ### Tenant sets: one wave over many databases or schemas A product with a database or schema per tenant applies the same migrations to every one of them. `environments..tenants` in `yodel.config.ts` makes the environment a tenant set: ```ts environments: { prod: { tenants: ["acme", "globex", "initech"] }, // or { file: "tenants.txt" }, one name per line or a JSON array, // or { query: "SELECT schema_name FROM information_schema.schemata WHERE schema_name LIKE 'tenant_%'" } }, waves: [ { env: "staging", gate: "on-destructive" }, { env: "prod", gate: "always", shares: 4 }, ], ``` The profile in `chant.config.ts` names one database (ClickHouse `databases`) or schema (Postgres `schemas`): the template the declared schema and the migrations are written against. A tenant is named `/`, as in `yodel apply prod/acme`, and is the environment's profile on the tenant's database or schema. Each statement and each pre-migration check is sent with the template's name replaced by the tenant's, wherever it stands alone as an identifier, quoted or not. An Op step runs on each tenant too: a backfill's `yodelSql` statements are renamed the same way, and a rebuild or a `PostgresMigrationOp` runs on the tenant's table, with the template renamed in its options and in the migration's recorded schema. Each tenant keeps its own history, `_`, its own lock, and its own backfill receipts. `yodel apply prod` without a tenant refuses. `yodel ci` renders the tenant set's wave with a run per tenant of a list or a file. A set given by a query is read when the wave runs, not when the pipeline is rendered: its wave is one run, or one per share (`prod/@1of4` to `prod/@4of4`), and each run plans every tenant the query lists at that moment whose name hashes to it. A tenant added since the pipeline was rendered is in the next run of the wave, and the rendered files do not change. The wave plans every tenant and gates once, on one approval of the set digest over every tenant's plan, which the deciding job prints. With `shares`, the wave is a deciding job and that many share jobs, each applying a slice of the tenants (for a list or a file, sorted by name). A share job plans its slice again, and a tenant whose plan moved since the decision (someone changed its schema, or a migration landed) is not applied: the job names it and exits 4, and applies the rest. For a set read by a query, a share whose tenants changed (a tenant added or gone, or a plan that moved) is not applied at all, and the job exits 4. Run the wave again to decide over the plan as it is now. A tenant requires what its environment requires: the same tenant when the environment before is a tenant set too, else that environment. `yodel plan prod` plans every tenant against its own history and shows them as one plan: the tenants with the same migrations pending once, as a group, then the wave and its approval. `--comment` posts it as one comment for the set, updated on the next push. `yodel approve prod` plans every tenant as the deciding job does, shows what is pending on each group, asks you to type `prod`, and approves the wave's set digest, then starts the waiting job again. `yodel approve prod/acme` approves one tenant's plan for `yodel apply prod/acme`, outside the pipeline. `yodel status prod` lists every tenant against its own history, then how many are up to date (exit 3 while any is behind), and `yodel report prod` shows a column per tenant. ### Steps around apply: checks before and after Checks a runbook asks for before a migration (replication lag, long transactions, disk headroom) and smoke queries after go in `environments..steps`: ```ts environments: { prod: { steps: { before: [ { name: "no-long-transactions", sql: "SELECT pid FROM pg_stat_activity WHERE xact_start < now() - interval '5 minutes'", onFailure: "approve" }, { name: "disk", command: "./scripts/disk-headroom.sh" }, ], after: [{ name: "smoke", sql: "SELECT count(*) FROM app.orders", expect: { min: 1 } }], }, }, }, ``` A `sql` step is a read-only query on the environment's server, judged like a migration's pre-migration check: `expect` is `empty` (the default, no rows), `true`, or `{ min, max }`. A `command` step runs with `sh -c` in the project directory, with `YODEL_ENV` and `YODEL_STEP` (`before` or `after`) set, and passes when it exits 0. The steps are read from `yodel.config.ts` at the base commit, like the policy, so a pull request cannot add a step that runs with the writer's credentials, or remove one that would hold it. They run only when something is pending: - `before` steps run under the lock just before the first migration. A SQL check also runs when the wave is planned (`--wave-plan`), and in the migrations Op's path `yodel apply ` runs it before the gate. A command runs once per apply, under the lock: it may have side effects (a notification, a snapshot), so planning only shows that it will run. In a wave it runs once the wave may apply, after the SQL checks. A failing step refuses the run, exit 4, nothing applied. - With `onFailure: "approve"`, a failing `before` step holds the wave for a person instead, whatever its gate: the wave plan's digest names the held steps, `on-destructive` waits on it, and under `never` the wave job applies nothing and exits 3 with the command that approves the wave's digest, `yodel approve --plan `. Once someone approves that plan, the job run again applies it. If the step passes by then, the plan has moved and the wave decides again. A command's failure is only known under the lock, so it is not in the plan's digest: the job stops there the same way, and the command runs again on the approved run, its failure let through. - `after` steps run under the lock once the last migration's statements succeeded. A failing one fails the run, exit 1; the migrations stay applied. Every outcome is in the history: the `before` steps on each migration's `started` row, the `after` steps on the last migration's `succeeded` row (`note.steps`). `yodel report` lists them as `step` entries in the audit log. ## Approval modes: which approvals count Anyone who can push to `chant/lifecycle` can record an approval, in any name. That is enough for dev. For an environment where it is not, a wave's `approval` says which approvals of its digest count: ```ts waves: [ { env: "dev", gate: "never" }, { env: "staging", gate: "always", approval: "pr-review" }, { env: "prod", gate: "always", approval: "sealed" }, ], ``` | `approval` | What counts | |---|---| | `ledger` (the default) | any approval of the digest (`yodel approve`), as above | | `pr-review` | an approving review of the merged pull request's head by a writer other than its author, when that head planned this digest; a `yodel approve` of the digest counts too | | `sealed` | only an approval sealed with an ssh key (`yodel approve --sign`) that the signers file at the base commit lists for the approver | The mode is read at the base commit with the gate, so a pull request that switches prod from `sealed` to `ledger` merges, and prod still counts only sealed approvals. `yodel ci` writes it into `yodel-waves.json`, where `chant run wave` reads it, and `yodel apply --wave-apply` checks it again under the lock. chant documents the modes in [Which approvals count](https://intentius.io/chant/guide/op-waves/#which-approvals-count). `yodel plan` and the pull request comment name the mode of a wave that waits, and a sealed wave's approve command ends in `--sign`. The history records how each wave was approved: the mode, and the review (pull request, head, approvers) or the signer, on the wave's `note.wave`. The migrations Op's gate, which `yodel apply ` goes through outside the waves, takes `environments..approval`, `ledger` or `sealed`, also read at the base commit. Without it, the environment takes `sealed` when its wave is sealed, else `ledger`. chant's gate step does not know the mode, so it may pass an unsealed approval, but the Apply step (`yodel apply --execute`) refuses it, exit 3, nothing applied, and says why. `pr-review` is a wave's only: the review approves the plans the pull request's pipeline records for the waves. ### Sealed approvals The signers file is `.chant/allowed_signers` at the root of the repository (or the path `.chant/trust.json` names), in ssh-keygen's allowed signers format: a principal and a public key per line. ```text alice ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAI... bob ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAI... ``` It is read at the base commit, never from the change being applied, so a key that a pull request adds to the file cannot approve that pull request's own plan: it counts for the changes merged after it. Changing the file is a change like any other, reviewed and merged. With no signers file at the base, no approval of a sealed gate counts. [chant workspace signers](https://intentius.io/chant/cli/workspace-signers/) documents how chant reads the file and rotates the keys in it. `yodel approve --sign` seals the approval with git's signing key (`user.signingkey`, with `gpg.format ssh`), or the private key `--key ` names. Under a sealed gate, `yodel approve` seals without being asked. The approver is `--actor `, else the name chant takes from `GITHUB_ACTOR`, `GITLAB_USER_LOGIN` or `USER`, and it must be the principal the signers file gives the key; a seal by a listed key in another approver's name does not count. With the command a waiting job printed: ```sh npx yodel approve prod --plan jcs1-sha256:... --sign --key ~/.ssh/id_ed25519 --actor alice ``` The job verifies the seal with `ssh-keygen`, which the CI image and `node:22-bookworm` carry. A job image without it cannot verify a seal, and the wave waits. ### Pull request review Under `pr-review`, the pull request's own review approves the wave, with no `yodel approve`. `yodel ci` renders a pull request job that runs `chant run wave --spec yodel-waves.json --record-plans`: it plans every wave at the pull request's head, as the wave will plan after the merge (each environment a wave requires taken as having applied what is pending, which the wave checks when it runs; no `before` step is run), and records the digests on `chant/lifecycle` (`_wave-plans/yodel-apply/.json`). It holds no writer: it plans with each environment's reader, and runs the write probe first ([Configuration](/sql-yodeler/configuration/)), as every pull request job does. | Forge | The job | What counts as an approving review | |---|---|---| | GitHub | `.github/workflows/yodel-apply-plans.yml` | the reviewer's newest review is `APPROVED` on the head commit, and the reviewer has write, maintain or admin on the repository | | Forgejo | `.forgejo/workflows/yodel-apply-plans.yml` | the same, and the review is official (Forgejo's mark for a reviewer with write access), not stale and not dismissed | | GitLab | `yodel-apply-record-plans`, in merge request pipelines, stage `plans` | the merge request's approvals, by members with the Developer role or higher, and only when the project removes approvals on a new push | After the merge, the wave finds the pull request that made the commit it applies, and its head. When a writer other than the author approved that head and the head recorded the digest the wave plans now, the wave applies and chant writes an approval to the ledger that names the review. The author's own review never counts, and a writer's standing request for changes holds the review back. A plan that moved after the review (another merge first, a table changed by hand, a wave before it that did not apply what was pending) has a new digest, so the wave waits for a `yodel approve` of it. The wave job reads the review with the forge's API: on GitHub and Forgejo with the job's token (`yodel ci` gives the wave's job `GITHUB_TOKEN`, and on GitHub the workflow `pull-requests: read`); on GitLab with `CHANT_FORGE_TOKEN` or `GITLAB_TOKEN`, a token with `read_api`. GitLab's limits: - An approval on GitLab names no commit, so it is tied to the head only by the project's setting "Remove all approvals when commits are added to the source branch" (`reset_approvals_on_push`). Without it, no GitLab approval counts. - The record job pushes `chant/lifecycle`, which `CI_JOB_TOKEN` can do only when the project allows Git push requests from job tokens. Otherwise the job fails, nothing is recorded, and the wave waits for `yodel approve`. - Merged results and merge trains plan a commit that is not the merge request's head; the recorded plan is the head's, so a merge that changes what the wave plans waits for `yodel approve`. A pull request from a fork runs its pipeline without the repository's secrets and with a read-only token: the record job can neither read the environments nor push `chant/lifecycle`, so nothing is recorded, and after the merge the wave waits for a `yodel approve` of its digest. Approve those as under `ledger`. ## The pull request comment `yodel plan --comment` posts the plan as one comment per environment on the pull (merge) request, and updates that comment on the next push rather than adding another. `--format markdown` prints the same Markdown without posting. From the ClickHouse example, for its rebuild: ````console $ npx yodel plan dev --format markdown ### yodel plan: `dev` ClickHouse, history `yodeler.history (single (default))`: 2 applied, **1 pending**. #### `20261010T1722-events-by-id` | # | Step | Rule | Class | Object | |---|---|---|---|---| | 0 | **Op ClickHouseRebuildOp** | SQLCH220 | rebuild | `shop.events` | Op steps: - step 0, `shop.events`: made by Op `ClickHouseRebuildOp` (SQLCH220 rebuild orderBy). yodel apply runs it as this step.
ClickHouseRebuildOp for events ```text needs a rebuild, which ALTER cannot make: SQLCH220 Change the sorting key on orderBy (( kind , at ) -> ( kind , id )): The sorting key is the on-disk order of every part (and in ReplacingMergeTree the deduplication identity); beyond appending new columns (SQLCH216) it cannot change in place. https://clickhouse.com/docs/sql-reference/statements/alter/order-by. Run it as the rebuild migration Op: ClickHouseRebuildOp({ table: "shop.events", ... }) from @intentius/chant-lexicon-sql/clickhouse ``` ```ts export const { op } = ClickHouseRebuildOp({ name: "rebuild-shop-events", env: "", table: "shop.events", dualWrite: { mode: "materialized-view", cutoverColumn: "at" } }); ```
#### Approval Plan digest the apply gate binds: `jcs1-sha256:82c892052401d2b5a35d49964395124736586c6a8222858f8f0905155d29eb47` Approve it with: ```sh chant approve migrate-dev approve-migrate-dev --plan jcs1-sha256:82c892052401d2b5a35d49964395124736586c6a8222858f8f0905155d29eb47 ``` The newest approval of `approve-migrate-dev`, by yodel at 2026-10-10T17:22:18.586Z, is for `jcs1-sha256:c6a49076c4a2b58c984b7a73d59e86c5788e82cb89728c5ec24307869e72bd03`, not this plan; it does not hold. What moved (as far as yodel can tell): history changed: applied since: 20261010T1722-add-country (by yodel@example at 2026-10-10 17:22:22.954144); pending migrations changed: committed to since approval: 20261010T1722-events-by-id (1880fda at 2026-10-10T11:22:28-06:00); live schema changed: as expected after the history change above: applying a migration changes the live schema, and yodel cannot rebuild the live schema as it was at approval, so it cannot confirm that nothing else in it changed. ```` The hidden `yodel:subject` line at the end records the state the plan was made from; the next `--comment` run reads it from the comment it updates, to say what moved. The forge, the repository, the request and the API come from the CI job's environment: | Forge | Detected by | Token | |---|---|---| | GitHub Actions | `GITHUB_ACTIONS=true` | `GITHUB_TOKEN`, with `pull-requests: write` | | GitLab CI | `GITLAB_CI=true` | `GITLAB_TOKEN`: a project or personal access token with `api` scope (`CI_JOB_TOKEN` cannot write merge request notes) | | Forgejo Actions | `FORGEJO_ACTIONS=true` or `GITEA_ACTIONS=true` (or a GitHub-style job whose API URL ends in `/api/v1`) | `FORGEJO_TOKEN`, else `GITHUB_TOKEN` | Outside those, or to override them: `--forge github|gitlab|forgejo`, `--repo`, `--pr`, `--api-url`, and `--token-env ` for the variable that holds the token. The token is never a flag and never printed. The comment job needs only read credentials for the database; the starter templates give it the reader. ## When the digest moves Anything the digest covers that changes after the approval (another migration, an edited one, another run applying first, a table changed out of band) makes a new digest, and the approval does not carry over. The gate asks again, nothing is applied, and yodel says what moved, as far as it can tell from the history and git. In the ClickHouse example a plan is approved, then someone adds a column by hand before the apply runs: ```console $ npx yodel plan dev Plan for dev (clickhouse 26.8.15.10 at 127.0.0.1:8123, history yodeler.history (single (default))): 4 applied, 1 pending Kept by earlier steps, until their retention date: shop.events__chant_old (until 2026-10-17T17:22:47.304Z). yodel cleanup dev drops each after its date, behind an approval. 20261010T1723-add-source 0 statement SQLCH201 metadata shop.events ALTER TABLE `shop`.`events` ADD COLUMN source LowCardinality(String) DEFAULT 'web' AFTER `country` Plan digest: jcs1-sha256:3c0f36706f2c91b7ac34fc175ae3a05ff0161364c71a6ca7dbecdc8982255544 Approve it with: chant approve migrate-dev approve-migrate-dev --plan jcs1-sha256:3c0f36706f2c91b7ac34fc175ae3a05ff0161364c71a6ca7dbecdc8982255544 or, at a terminal, with npx yodel approve dev, which shows the plan and asks you first. The newest approval of approve-migrate-dev, by yodel at 2026-10-10T17:22:58.009Z, is for jcs1-sha256:96f8f3625b2628eedd9a5567254b9b7ecf5e53ae0a22550d6da7cf3a37ff200e, not this plan; it does not hold. What moved: history changed: applied since: 20261010T1722-fill-country (by yodel@example at 2026-10-10 17:23:03.136317); pending migrations changed: new pending migration: 20261010T1723-add-source. ``` ```console $ npx chant approve migrate-dev approve-migrate-dev --plan jcs1-sha256:3c0f36706f2c91b7ac34fc175ae3a05ff0161364c71a6ca7dbecdc8982255544 Gate "approve-migrate-dev" on "migrate-dev" resolved by yodel at 2026-10-10T17:23:08.576Z This approves the plan jcs1-sha256:3c0f36706f2c91b7ac34fc175ae3a05ff0161364c71a6ca7dbecdc8982255544, and only that plan. A run whose fresh plan differs refuses rather than applying it. This records the resolution as a fact; it does not itself re-run anything. Run `chant run migrate-dev` and it walks through gate "approve-migrate-dev". ``` ```sql ALTER TABLE shop.events ADD COLUMN debug String DEFAULT '' ``` ```console $ npx yodel apply dev ... Running migrate-dev (/clickhouse/ops/migrate-dev.op.ts) [phase] Plan ✓ shellCmd(cmd=yodel apply dev --digest) 1.2s [phase] Approve • gate:approve-migrate-dev() skipped [refused] Gate "approve-migrate-dev" is approved, but not for this plan. approved: jcs1-sha256:3c0f36706f2c91b7ac34fc175ae3a05ff0161364c71a6ca7dbecdc8982255544 (by yodel at 2026-10-10T17:23:08.576Z); planned: jcs1-sha256:4646f73faf35fc9a8563f817637d72126db7718931d03158677fb5160fe33605. The configuration or the live system changed between that approval and this plan, so it needs a fresh one: chant approve migrate-dev approve-migrate-dev [phase] Apply • shellCmd(cmd=yodel apply dev --execute, env={"YODEL_APPROVED_PLAN":{"kind":"step-output-ref","step":"plan","path":"stdout"}}) skipped Op "migrate-dev" is gated on "approve-migrate-dev" after 2.0s plan : jcs1-sha256:4646f73faf35fc9a8563f817637d72126db7718931d03158677fb5160fe33605 approve : chant approve migrate-dev approve-migrate-dev --plan jcs1-sha256:4646f73faf35fc9a8563f817637d72126db7718931d03158677fb5160fe33605 expires : 2026-10-12T17:23:12.217Z Nothing was applied. The approval of jcs1-sha256:3c0f36706f2c91b7ac34fc175ae3a05ff0161364c71a6ca7dbecdc8982255544 (by yodel at 2026-10-10T17:23:08.576Z) does not hold: the plan moved since. What moved, as far as yodel can tell: live schema changed: no state yodel can rebuild from the history and git hashes to the approved digest with today's live schema, so an object in the environment's live schema (a table, view, index, dictionary, database or schema) changed since approval (or a part yodel cannot rebuild did). migrate-dev is waiting at gate "approve-migrate-dev" for approval of this plan (jcs1-sha256:4646f73faf35fc9a8563f817637d72126db7718931d03158677fb5160fe33605). Approve it with: chant approve migrate-dev approve-migrate-dev --plan jcs1-sha256:4646f73faf35fc9a8563f817637d72126db7718931d03158677fb5160fe33605 or, at a terminal, with npx yodel approve dev, which shows the plan and asks you first; then run npx yodel apply dev again. [exit 3] ``` Dropping the column puts the live schema back, and the digest is the approved one again. The gate still asks again, because the refused run asked for approval after that approval was given; an approval holds only when it is newer than the latest request. `yodel plan` and `yodel apply` say so: ```console $ npx yodel plan dev Plan for dev (clickhouse 26.8.15.10 at 127.0.0.1:8123, history yodeler.history (single (default))): 4 applied, 1 pending Kept by earlier steps, until their retention date: shop.events__chant_old (until 2026-10-17T17:22:47.304Z). yodel cleanup dev drops each after its date, behind an approval. 20261010T1723-add-source 0 statement SQLCH201 metadata shop.events ALTER TABLE `shop`.`events` ADD COLUMN source LowCardinality(String) DEFAULT 'web' AFTER `country` Plan digest: jcs1-sha256:3c0f36706f2c91b7ac34fc175ae3a05ff0161364c71a6ca7dbecdc8982255544 Approve it with: chant approve migrate-dev approve-migrate-dev --plan jcs1-sha256:3c0f36706f2c91b7ac34fc175ae3a05ff0161364c71a6ca7dbecdc8982255544 or, at a terminal, with npx yodel approve dev, which shows the plan and asks you first. The newest approval of approve-migrate-dev, by yodel at 2026-10-10T17:23:08.576Z, is for this plan's digest, but approval was asked for again after it at 2026-10-10T17:23:12.217Z (a run holding jcs1-sha256:4646f73faf35fc9a8563f817637d72126db7718931d03158677fb5160fe33605), so it does not hold; approve this plan again. ``` ```console $ npx yodel apply dev ... Op "migrate-dev" is gated on "approve-migrate-dev" after 1.7s plan : jcs1-sha256:3c0f36706f2c91b7ac34fc175ae3a05ff0161364c71a6ca7dbecdc8982255544 approve : chant approve migrate-dev approve-migrate-dev --plan jcs1-sha256:3c0f36706f2c91b7ac34fc175ae3a05ff0161364c71a6ca7dbecdc8982255544 expires : 2026-10-12T17:23:19.304Z Nothing was applied. The approval of jcs1-sha256:3c0f36706f2c91b7ac34fc175ae3a05ff0161364c71a6ca7dbecdc8982255544 (by yodel at 2026-10-10T17:23:08.576Z) is for this plan's digest, but approval was asked for again after it at 2026-10-10T17:23:12.217Z (a run holding jcs1-sha256:4646f73faf35fc9a8563f817637d72126db7718931d03158677fb5160fe33605), so it does not hold; approve this plan again. migrate-dev is waiting at gate "approve-migrate-dev" for approval of this plan (jcs1-sha256:3c0f36706f2c91b7ac34fc175ae3a05ff0161364c71a6ca7dbecdc8982255544). Approve it with: chant approve migrate-dev approve-migrate-dev --plan jcs1-sha256:3c0f36706f2c91b7ac34fc175ae3a05ff0161364c71a6ca7dbecdc8982255544 or, at a terminal, with npx yodel approve dev, which shows the plan and asks you first; then run npx yodel apply dev again. [exit 3] ``` After a new approval (`yodel approve`), the apply goes through. A migration that failed part way moves the digest too, since the statements that succeeded are part of it; the Postgres example approves again before resuming. ## Policy and overrides Some rules are not lint: no drops in production, no rebuilds during business hours, every silence cites a ticket. `yodel.config.ts` declares them as `policy`, rules over the plan `yodel apply` would run: ```ts // yodel.config.ts import { defineConfig } from "@intentius/sql-yodeler"; export default defineConfig({ policy: { overriders: ["alice", "bob"], rules: [ { id: "no-drops-in-prod", environments: ["prod"], description: "no drops in prod", refuse: { classes: ["drop"], destructive: true } }, { id: "silences-cite-a-ticket", silences: { reason: /\b[A-Z]+-\d+\b/ } }, { id: "no-rebuilds-in-hours", environments: ["prod"], check: (plan) => { const hour = new Date(plan.now).getUTCHours(); const rebuild = plan.pending.some((m) => m.steps.some((s) => s.classes.includes("rebuild"))); return rebuild && hour >= 8 && hour < 18 ? "a rebuild between 08:00 and 18:00 UTC" : undefined; }, }, ], }, }); ``` A rule applies to the environments it lists, or to every one. It denies the plan when any of its parts does: - `refuse.classes` denies a pending step of one of those classes (ClickHouse: `metadata`, `create`, `rewrite`, `rebuild`, `drop`; Postgres: `metadata`, `create`, `validate`, `concurrently`, `rewrite`, `expand`, `drop`), and `refuse.destructive` any destructive step. - `silences.reason` denies a `-- yodel:allow` silence (of `silences.rules`, default any) whose reason does not match the pattern. - `check` is a TypeScript predicate over the plan: the environment, the dialect, the time (`now`), and each pending migration with its id, checksum, steps (classes, destructive, rules, SQL, Op) and silences. A message, a list of messages or `false` denies; `undefined`, `true` or `[]` allows. The policy is read from `yodel.config.ts` at the base commit, the commit a change merges onto (the merge base with the target branch on a pull request, `HEAD^1` on main, or `YODEL_BASE`), as a wave's gate is, never from the working tree. A pull request that weakens or deletes a rule is judged by the rule the branch had, and its own policy applies once it is merged. A rule the base does not have applies to nothing yet. When the working tree's policy differs from the base's, `yodel plan` and `yodel apply` say which one applies. `yodel plan` shows each rule's outcome, and the pull request comment shows them in a table with the command that overrides a denial. `yodel apply` judges the policy before the gate and the Apply step judges it again under the lock: a rule that denies the plan refuses the apply with exit 4, naming the rule and why, and nothing is applied. An override is a person's decision, with a reason, to apply a plan a rule denies: ```sh npx yodel override prod --rule no-drops-in-prod --reason "the table has been empty since OPS-412" --by alice ``` It is recorded on chant's gate ledger (the `chant/lifecycle` branch) as an approval of the gate `override-` of the environment's migrations Op, for exactly the current plan digest, with the reason as its note (`chant approve override- --plan --note `). It counts only for that digest: if the plan moves (a migration added or edited, the live schema changed), the rule denies again and needs a new override. It counts only when `policy.overriders` lists who recorded it (`--by`, else chant's actor: the CI user or your login). `yodel override` refuses a rule that does not deny the current plan, so an override always answers one denial. An override does not approve the plan: the migrations Op's gate still asks for approval as before. The Apply step writes the overrides it relied on into the history, in the `note` of each migration's `started` row, as `{"policy":{"overrides":[{"rule":...,"by":...,"at":...,"reason":...,"digest":...}]}}`. ### Reverts `yodel revert` ([Reverting](/sql-yodeler/migrations/#reverting-yodel-revert)) is judged by the same policy, read at the base commit, before the gate and again under the lock. The rules see the revert as a plan with one entry in `pending`, the migration reverted, whose steps are the revert's statements in the order they run (a hand-written revert step's first), and `plan.revert` names the migration and the one it goes back to. A revert usually drops what the migration added, so a rule that refuses drops (or `refuse.destructive`) in prod refuses the revert too: exit 4, nothing reverted. To revert anyway, override for the revert's own digest: ```sh npx yodel override prod --revert 20261010T0900-add-coupons --rule no-drops-in-prod --reason "added by mistake, OPS-415" --by alice ``` With `--step ` as well when the revert has a hand-written step. An override of the apply's plan never counts for a revert, nor the other way round. The revert's Apply step writes the overrides it relied on into the note of the migration's `reverted` row. A revert always stops at the migrations Op's gate, whatever the wave's gate policy in an environment `waves` names: under `on-destructive`, a destructive revert waits for an approval, and so does any other revert. `yodel revert --json` says what the wave's gate, at the base commit, makes of the revert (`wave`: its destructive statements, counted as a wave plan counts them, and whether the policy asks for an approval). ## Proven by The scenario claims below run what this page describes against a real server, once plain (it passes) and once with the behaviour broken (the claim catches it). [Claims status](/sql-yodeler/claims/) lists every claim. | Claim | What it says | Plain, broken | Last run | |---|---|---|---| | `approval` | a pending migration applies only after chant approve of its plan digest, and the history records that digest; a failing pre-migration check refuses it, and so does a policy rule, read at the base commit, unless an override is recorded for that digest, for a yodel revert as for an apply; the audit log derived from the history and the ledger accounts for every approval and apply, and names one removed; a reader and a writer that are one user are refused, and the reader, its password a minted token, cannot write; an approval is refused once the migration changed after it, and nothing is applied | ClickHouse: pass, caught; Postgres: pass, caught | `c6f58a4`, 2026-10-10 | | `pr-comment` | the pull request comment lists the pending migrations with each statement's class, and the digest the gate asks for | ClickHouse: pass, caught; Postgres: pass, caught | `c6f58a4`, 2026-10-10 | | `waves` | the apply pipeline runs one wave per environment, in order, each behind its gate policy read from the base commit, applies only what the wave before applied, and applies a tenant set's migrations to every tenant behind one gate; a sealed wave counts only an approval sealed by a signer the base commit lists | ClickHouse: pass, caught; Postgres: pass, caught | `c6f58a4`, 2026-10-10 | --- # Reports and the audit log Source: https://intentius.io/sql-yodeler/audit/ ## Optional: hand this page to your coding agent ```text Following https://intentius.io/sql-yodeler/audit/, fetch the `chant/lifecycle` branch and run `npx yodel report` with the readers' credentials. Tell me which migrations are applied where, who approved each apply, and each gap the report lists, with what the page says it means. Never run `yodel apply` against a shared environment, never run `chant approve`, `yodel approve` or `yodel override`, never edit the `chant/lifecycle` branch or `.chant/allowed_signers`, never merge; approvals and applies belong to people. ``` Each environment's history table says what ran there, and chant's gate ledger (the `chant/lifecycle` branch) says who approved which plan. `yodel report` reads both for every environment and joins them: a table of each migration's state in each environment, an audit log, and the gaps the two sources show. ```text $ yodel report Report for /work/shop (clickhouse, 2 environments) migration dev prod 20261010T0900-init applied 2026-10-10 09:02 applied 2026-10-10 11:40 20261010T1000-add-note applied 2026-10-10 10:05 pending Gaps: none. Every applied migration has its approval on the ledger, and no history row is missing. Audit (7 entries): 2026-10-10T09:01:12.410Z dev approval-requested jcs1-sha256:3e1b0c9a4f... (run 1c7e...) 2026-10-10T09:01:40.002Z dev approved by alice jcs1-sha256:3e1b0c9a4f... 2026-10-10T09:02:03.118204Z dev applied 20261010T0900-init by ci@runner-7 jcs1-sha256:3e1b0c9a4f... ... ``` By default it reads every environment in `chant.config.ts`'s `sql.profiles`, in that order; name some to read only those (`yodel report dev prod`). In CI, fetch the `chant/lifecycle` branch first so the ledger is there. It reads only and takes no lock. ## The audit log Every entry comes from a history row or a ledger line, so the log cannot say something they do not. Deriving it again from the same sources gives the same entries with the same ids. | Entry | From | |---|---| | `approval-requested` | a pending fact on the ledger: a run stopped at the migrations Op's gate, or the environment's wave stopped at its wave gate | | `approved` | a resolution of the gate or the wave gate (`chant approve`, `yodel approve`) | | `override` | a resolution of an `override-` gate (`yodel override`), with the rule and the reason | | `silenced` | a `-- yodel:allow` silence, on the migration's first started row | | `applied` | the history row that first recorded the migration succeeded | | `refused` | a failed migration row naming a pre-migration check | | `failed` | any other failed migration row: a statement, an Op step | | `repaired` | a succeeded row after the migration was applied (`yodel repair`) | `--audit audit.jsonl` keeps the log as a file, one entry per line. Each run appends the entries the file does not have yet and never rewrites a line. `--audit audit.jsonl --check` appends nothing. Instead it names every entry of the file that the sources no longer give: a ledger line or a history row removed since the file was written. A CI job that keeps `audit.jsonl` as an artifact, or commits it, can run `--check` before it appends. ## Gaps `yodel report` exits 4 and lists each gap: - A history row missing. Each environment's history numbers its rows from 1 with no hole, because every run appends at the highest `seq` plus one, under the lock. A hole is a row someone deleted. - An applied migration whose plan digest has no approval of the environment's gate on the ledger: the approval was removed, or the migration was applied without one. A baseline recorded by `yodel init --from` has no digest and needs no approval; one recorded by `yodel init --baseline` carries the digest its gate approved. A migration a wave of the apply pipeline applied ([waves](/sql-yodeler/workflows/)) was let through on the wave's gate, `yodel-apply-wave-` on the `yodel-apply` ledger, for the wave's set digest, not on the migrations Op's gate. The wave's apply records that gate and digest in the migration's `started` history row (`note.wave`), and the report looks for the approval there. A wave whose gate policy at the base commit needed no approval (`never`, or `on-destructive` with nothing destructive) records that instead, checked under the lock when it applied, and is not a gap. - With `--audit --check`, an entry of the file the sources no longer give. An environment that cannot be read (its server does not answer) is reported with its error, and the exit is 1 when there is no gap. ## The run view `--html report.html` writes a self-contained page with no external resources, safe to upload as a CI artifact. It has one row per migration (each merge that added one) and one column per environment, in the order they were read, so a change's progress from dev to prod reads left to right. Each cell shows the state, when, who applied it and who approved its digest. The gaps and the audit log follow. ## The JSON `yodel report --json` prints the report described by `schemas/report.schema.json`. Within version 1 fields are only added. | Field | What it holds | |---|---| | `version` | 1 | | `generatedAt` | when the report was read, ISO 8601 UTC | | `project` | the project directory | | `dialect` | `clickhouse` or `postgres` | | `migrations` | the migrations in chain order, then any an environment applied that the directory no longer has | | `environments` | one per environment read, in order | | `environments[].name` | the environment | | `environments[].history` | its history table, `.history` | | `environments[].op` | its migrations Op: `name` and `gate` | | `environments[].error` | why it could not be read | | `environments[].notes` | what was read only in part, such as a ledger that could not be read | | `environments[].migrations` | each migration's state there | | `environments[].migrations[].id` | the migration | | `environments[].migrations[].state` | `applied`, `pending`, `failed` or `started` (stopped part way) | | `environments[].migrations[].inDirectory` | whether the migrations directory still has it | | `environments[].migrations[].at` | when it was applied; for failed or started, its latest row's time | | `environments[].migrations[].by` | who applied it | | `environments[].migrations[].digest` | the plan digest it was applied under | | `environments[].migrations[].approvedBy` | who approved that digest on the ledger | | `environments[].migrations[].approvedAt` | when | | `audit` | every entry, oldest first | | `audit[].id` | a hash of the entry and its source | | `audit[].at` | when, ISO 8601 UTC | | `audit[].environment` | the environment | | `audit[].kind` | one of the entries in the table above | | `audit[].by` | the approver, or who ran the apply | | `audit[].migration` | the migration, for history entries | | `audit[].digest` | the plan digest | | `audit[].detail` | the silence, the override's rule and reason, the error, the repair's reason | | `audit[].wave` | on an `applied` entry a wave applied: the wave's gate, its set digest, and `approved` or `not-required` | | `audit[].wave.gate` | the wave's gate, `yodel-apply-wave-` | | `audit[].wave.digest` | the wave's set digest, the one its approval is bound to | | `audit[].wave.status` | `approved`, or `not-required` when the wave's policy at the base needed no approval | | `audit[].source` | the ledger file, Op and gate; or the history table, the row's `seq` and its run | | `gaps` | each gap | | `gaps[].environment` | the environment | | `gaps[].kind` | `history-row` or `approval` | | `gaps[].detail` | what is missing, in a sentence | | `auditFile` | with `--audit`: the file | | `auditFile.path` | the file as given | | `auditFile.appended` | how many entries were appended (0 with `--check`) | | `auditFile.missing` | the entries of the file the sources no longer give, shaped as `audit[]` | ## What proves it The `approve-and-apply` claim, after its approval, pre-migration check and policy parts, runs `yodel report --json --audit audit.jsonl`. Every applied migration must have its `applied` and `approved` entries under the same digest, with no gap, and `--check` must then pass. Under `BREAK=1` the approval line is removed from the ledger before `--check`, which must name the missing entry and the unapproved migration. Unit tests (`test/report.test.ts`) remove a history row and a ledger line from fixed sources and find each as a gap. They also check that every field in the table above is in the schema. ## Proven by The scenario claims below run what this page describes against a real server, once plain (it passes) and once with the behaviour broken (the claim catches it). [Claims status](/sql-yodeler/claims/) lists every claim. | Claim | What it says | Plain, broken | Last run | |---|---|---|---| | `approval` | a pending migration applies only after chant approve of its plan digest, and the history records that digest; a failing pre-migration check refuses it, and so does a policy rule, read at the base commit, unless an override is recorded for that digest, for a yodel revert as for an apply; the audit log derived from the history and the ledger accounts for every approval and apply, and names one removed; a reader and a writer that are one user are refused, and the reader, its password a minted token, cannot write; an approval is refused once the migration changed after it, and nothing is applied | ClickHouse: pass, caught; Postgres: pass, caught | `c6f58a4`, 2026-10-10 | --- # Drift Source: https://intentius.io/sql-yodeler/drift/ ## Optional: hand this page to your coding agent ```text The drift watch reported drift in the environment I name. Following https://intentius.io/sql-yodeler/drift/, run `npx yodel drift ` with the reader's credentials and tell me what changed out of band. Then ask me whether to put the live object back by hand or keep the change. To keep it, declare it in src/, run `npx yodel new ` and `npx yodel lint`, and open a pull request. Never run `yodel apply` against a shared environment, never run `chant approve`, `yodel approve` or `yodel override`, never edit the `chant/lifecycle` branch or `.chant/allowed_signers`, never merge; approvals and applies belong to people. ``` Drift is a declared object changed out of band (by hand, by another tool) or gone. `yodel drift ` compares the declared objects with the live ones, through chant (`chant lifecycle diff --live`). An object the project does not declare is never reported. What the server is compared with (`--against`): - `history`, the default when the project has migrations: the schema recorded by the newest migration the environment's history records as applied (`migration.json`'s `schema`). A migration merged but not applied yet, waiting at its gate or refused, is not drift; a change made by hand is. With no migration applied yet there is no recorded schema, and nothing is reported. - `src`, the default without migrations: the declared schema in `src/`. Exit codes: 0 no drift, 2 drift, 1 the drift could not be read, or a declared object could not be read (its state is then unknown, not clean). `--json` prints the report as JSON, with what it was compared with (`against`) and a `fingerprint` of the drift: a hash of the environment and what drifted, the same for the same drift, which the tracking issue uses to tell an unchanged drift from a new one. In the ClickHouse example, someone changes a column default by hand: ```sql ALTER TABLE shop.events MODIFY COLUMN source LowCardinality(String) DEFAULT 'app' ``` ```console $ npx yodel drift dev Drift in dev (clickhouse 26.8.15.10 at 127.0.0.1:8123) Compared with the schema 20261010T1723-add-source records, the newest migration the history records as applied. Changed out of band: events (ClickHouse::Table) column source default.expr: declared 'web', live 'app' [exit 2] ``` Put back, there is none: ```console $ npx yodel drift dev Drift in dev (clickhouse 26.8.15.10 at 127.0.0.1:8123) Compared with the schema 20261010T1723-add-source records, the newest migration the history records as applied. No drift: every declared object matches the server. ``` The output quoted here is from the examples' runs. ## On a schedule: the WatchOp `ops/watch-.op.ts` declares the same check as a chant `WatchOp` with a cron schedule: ```ts import { WatchOp } from "@intentius/chant/op"; // Every hour at :17, read dev and report drift: a declared object changed out // of band, or gone. Objects src/ does not declare are never reported. // `chant run watch-dev` runs it once, by hand. export const { op } = WatchOp({ name: "watch-dev", env: "dev", schedule: "17 * * * *" }); ``` The watch job in the template's pipelines runs it on its schedule (`chant run watch-dev`, underneath). A run takes a snapshot, diffs declared against live, and reports: ```console $ npx chant run watch-dev Snapshot saved to chant/lifecycle (sql(2)) sql — environment: dev 0 missing, 0 orphan, 0 disappeared, 0 newly observed, 0 drifted, 2 unchanged -------------------------------------------------------------------------------- sql (properties) 1 property drift across 1 resource(s), 0 accepted, 1 unchanged -------------------------------------------------------------------------------- PROPERTY DRIFT (declared vs live; baseline shown where one exists): - events (ClickHouse::Table) columns[4].default.expr: 'web' → 'app' [from: authored] [phase] Snapshot ✓ lifecycleSnapshot(env=dev) 923ms [phase] Diff ✓ lifecycleDiff(env=dev, live=true) 781ms [outcome] Drift=true Op "watch-dev" completed in 1.8s ``` `chant run` exits 0 when the watch completes, drift or not. The drift is the run's `Drift` outcome, in its run record on the local `chant/lifecycle` branch. The WatchOp compares with `src/`, so in the starter templates' watch job it is followed by `yodel drift --issue`, which compares with the history, fails the job on drift, and keeps the tracking issue below. The starter templates render a scheduled pipeline per watch for each forge (`npm run ci` writes them from `ops/`): a GitHub Actions or Forgejo Actions workflow on the watch's cron, and a GitLab CI job run by a pipeline schedule you create with the cron the file notes (GitLab keeps no cron in the file). The job reads with the reader's credentials and never pushes: its snapshot and run record stay in the job. Its token writes only issues: GitHub's job token gets `issues: write`, Forgejo's can write already, and GitLab uses `GITLAB_TOKEN`, the project access token the plan comment uses. ## The tracking issue `yodel drift --issue` keeps one issue for the project and environment on the forge, found again by a marker in its body (``, where `` is the project's directory in its repository, `.` at the root): - drift found and no issue open: it opens one, titled with the environment and the objects (`Drift in dev: events, daily`), its body the report as markdown, what it was compared with, the CI run that found it and the fingerprint; - drift found and the issue open: it updates the issue when the fingerprint changed, and leaves it when it did not; - no drift: it comments that the drift is gone and closes the issue. Drift found again later opens a new one; - the drift could not be read, or a declared object could not be (exit 1): it leaves the issue as it is, and says so. The forge, repository and token come from the CI job's environment, as for `yodel plan --comment` ([Approval](/sql-yodeler/approval/)): `GITHUB_TOKEN` on GitHub Actions, `GITLAB_TOKEN` on GitLab CI, `FORGEJO_TOKEN` or `GITHUB_TOKEN` on Forgejo Actions. `--forge`, `--repo`, `--api-url` and `--token-env` set them outside CI. `yodel drift` prints what it did to the issue on stderr, and `--json` adds it as `issue` (`opened`, `updated`, `unchanged`, `closed`, `none` or `left`, with the number and link). An issue the forge refuses fails the run with exit 1 unless it found drift (exit 2). The Postgres example's drift and watch are in its section 8. ## Proven by The scenario claims below run what this page describes against a real server, once plain (it passes) and once with the behaviour broken (the claim catches it). [Claims status](/sql-yodeler/claims/) lists every claim. | Claim | What it says | Plain, broken | Last run | |---|---|---|---| | `drift` | yodel drift reports a declared object changed out of band, naming the property, and one dropped; on the versioned path it compares with the newest applied migration's recorded schema, so a pending migration is not drift | ClickHouse: pass, caught; Postgres: pass, caught | `c6f58a4`, 2026-10-10 | --- # Installing and configuring Source: https://intentius.io/sql-yodeler/configuration/ ## Installing `@intentius/sql-yodeler` is on npm. It needs Node 22.12 or later, and the project also needs chant and the `sql` lexicon, which `chant.config.ts` and the declarations import: ```sh npm install @intentius/sql-yodeler @intentius/chant@0.122.0 @intentius/chant-lexicon-sql@0.122.0 npx yodel --version ``` No token is needed, on your machine or in CI. A project made from a starter template has all three in its `package.json` already, so `npm install` there is enough; [Your first migration](/sql-yodeler/getting-started/) goes from there. The starter templates' `package.json` names `^0.5.2`. While the version is 0.x, a caret range stays within one minor version, so moving to the next minor release means changing the range (`npm install @intentius/sql-yodeler@0.6`). To try a commit that is not released yet, from git or from a clone, see CONTRIBUTING.md. ## A project A project is one directory: ``` chant.config.ts the dialect and one profile per environment (chant's config) yodel.config.ts optional: what yodel needs per environment that chant's config does not say src/ the declared schema: chant's sql lexicon, in TypeScript or plain .sql files migrations/ the versioned migrations, written by yodel new (absent on the declarative path) ops/ the Ops yodel apply runs (a migrations Op or an ApplyOp per environment), and drift watches package.json ``` Every command takes `-C ` (`--dir`) for a project elsewhere than the working directory. ## chant.config.ts The connection and the scope of each environment are chant's: `sql.profiles.`. From the Postgres example: ```ts import type { ChantConfig } from "@intentius/chant/config"; import "@intentius/chant-lexicon-sql"; export default { lexicons: ["sql"], sourceDir: "src", sql: { dialect: "postgres", profiles: { dev: { url: process.env.DEV_POSTGRES_URL ?? "postgres://127.0.0.1:5432/postgres", user: { env: "DEV_POSTGRES_USER" }, password: { env: "DEV_POSTGRES_PASSWORD" }, schemas: ["shop"], }, }, }, } satisfies ChantConfig; ``` | Field | Meaning | |---|---| | `url` | ClickHouse: the HTTP interface (`http://clickhouse:8123`). Postgres: a connection URL with no password in it. | | `user`, `password` | `{ env: "" }`: the variable each is read from. Credentials are never written in the file. A `password` can also be a token source; see [Credentials](#credentials-a-reader-and-a-writer). | | `databases` (ClickHouse) | the databases the project's schema lives in. `yodel init --from` needs it, and the history's database must not be one of them. | | `schemas` (Postgres) | the same, for schemas. | | `topology` (ClickHouse) | the topology chant renders for; see [Topology](/sql-yodeler/topology/). | | `lockTimeoutMs`, `statementTimeoutMs`, `scanTimeoutMs` (Postgres) | the `lock_timeout` of every statement `yodel apply` sends, and of the live-schema reads before it and in `yodel drift`, which name the lock they waited on (default 5000), the `statement_timeout` of a catalog change (default 60000), and of a statement that reads or rewrites rows (default none). | ## Credentials: a reader and a writer Each environment has two database users. The reader can only read: pull request jobs, the drift watch and anyone at a terminal use it. The writer applies migrations, and only the environment's wave job after a merge holds it. `credentials()` gives a profile both: ```ts import { credentials } from "@intentius/sql-yodeler"; profiles: { dev: { url: "postgres://127.0.0.1:5432/app", ...credentials("dev", "postgres"), schemas: ["shop"] }, } ``` A process connects as the reader unless `YODEL_CREDENTIALS` is `writer` (every environment) or `writer:` (that one); only ``'s wave job sets `writer:`. Each role reads its user and password from a variable, by default `__READER_USER`, `__READER_PASSWORD`, `__WRITER_USER` and `__WRITER_PASSWORD` (`DEV_POSTGRES_READER_USER`), the names `yodel ci` maps the forge's secrets to. The third argument replaces a role's user or password: `credentials("prod", "postgres", { writer: { user: { env: "APP_MIGRATOR" } } })`. The starter templates' profiles are made this way. Every command refuses the project, exit 4, when an environment's reader and writer are one identity: both read their user from the same variable, the two variables hold the same user name, or both read their password from the same variable. A job holding such a reader could write. A command that writes (`yodel apply`, `yodel revert`, `yodel init`) refuses, exit 4, before its gate and before it connects, when it would run as an environment's reader: ```text yodel apply: refused: npx yodel apply dev writes, and this process holds dev's reader, DEV_CLICKHOUSE_READER_USER (reader), because YODEL_CREDENTIALS is not set. Run it as the writer: YODEL_CREDENTIALS=writer:dev npx yodel apply dev. Nothing was done. ``` `yodel apply --plan` and `--digest`, and `yodel revert --dry-run`, only read, and run as the reader. Each command yodel prints for a person to run that writes (`then run ... again` after a gate stop, `yodel approve`'s `Apply it with`, `yodel override`'s) starts with `YODEL_CREDENTIALS=writer:` for a profile made with `credentials()`. A server that refuses a write for want of a privilege (a profile without `credentials()`, or a writer missing a grant) is reported the same way, in one line naming the user and the server's message, exit 4. ```text yodel status: refused: sql.profiles.prod: the reader (PROD_POSTGRES_READER_USER) and the writer (PROD_POSTGRES_WRITER_USER) are the same user, app, so a job that holds the reader can write. Give each role its own database user. ``` ### Short-lived tokens A role's password can be minted when a connection needs it, so CI holds no long-lived database password. chant's `sql` lexicon mints it: | `password` | Mints with | Dialect | |---|---|---| | `{ token: "rds-iam" }` | `aws rds generate-db-auth-token`, for the profile's host, port and user; `region` or `AWS_REGION` | Postgres | | `{ token: "cloud-sql-iam" }` | `gcloud sql generate-login-token` | Postgres | | `{ token: "entra" }` | `az account get-access-token` for Azure Database for PostgreSQL | Postgres | | `{ token: "command", command: ["./mint.sh"] }` | any program that prints the password on its standard output, run without a shell | both | ```ts prod: { url: "postgres://db.abc.us-east-1.rds.amazonaws.com:5432/app", ...credentials("prod", "postgres", { writer: { password: { token: "rds-iam" } } }), schemas: ["shop"], }, ``` A token is reused until 80% of its lifetime has passed (`ttlSeconds`; 900 for `rds-iam`, 3600 for `cloud-sql-iam` and `entra`, 300 for `command`), and minted again when the server refuses it, so a long apply never sends an expired one. A source that cannot mint fails the command as `no-credentials`, naming the source and the program but never what it printed. The cloud sources run the cloud's own command line, which reads the identity of the machine or the job; the `command` source is how a test, or a mint the others do not cover, supplies one. `yodel ci` gives each job the variables of the role it holds and nothing for a token-minted password. A job holding a role minted by `rds-iam`, `cloud-sql-iam` or `entra` also gets an OIDC token from the forge: | Forge | What `yodel ci` renders | |---|---| | GitHub | `permissions: id-token: write` on the workflow (the pull request, apply or watch workflow whose jobs hold such a role) | | GitLab | `id_tokens` on each job holding such a role: `AWS_ID_TOKEN` (audience `sts.amazonaws.com`), `GCP_ID_TOKEN` (`https://iam.googleapis.com/`, or the provider's own audience with `ci.login.gcp`) or `AZURE_ID_TOKEN` (`api://AzureADTokenExchange`) | | Forgejo | nothing: Forgejo gives a job no OIDC token, so the runner's own cloud identity (an instance profile, a workload or managed identity) mints. The rendered files say so. | The cloud's command line then has to reach that identity in the job. A runner that carries the identity itself needs nothing more. Otherwise `ci.login` in yodel.config.ts tells `yodel ci` how to exchange the token, and each job holding such a role logs in before yodel runs: ```ts ci: { forges: ["github", "gitlab"], login: { aws: { roleArn: "arn:aws:iam::123456789012:role/yodel-dev" }, gcp: { workloadIdentityProvider: "projects/123456789/locations/global/workloadIdentityPools/ci/providers/forge", serviceAccount: "yodel@shop.iam.gserviceaccount.com" }, azure: { clientId: "", tenantId: "" }, environments: { prod: { aws: { roleArn: "arn:aws:iam::123456789012:role/yodel-prod" } } }, }, }, ``` | Cloud (token source) | The login step | |---|---| | `aws` (`rds-iam`) | the token in a file, and `AWS_ROLE_ARN`, `AWS_WEB_IDENTITY_TOKEN_FILE` and `AWS_ROLE_SESSION_NAME` (`sessionName`, default `yodel`), which the aws command line assumes the role with | | `gcp` (`cloud-sql-iam`) | the token in a file, a workload identity credential configuration over it that impersonates `serviceAccount`, `gcloud auth login --cred-file` with it, and `GOOGLE_APPLICATION_CREDENTIALS` | | `azure` (`entra`) | the token in a file, `az login --service-principal --federated-token`, and `AZURE_CLIENT_ID`, `AZURE_TENANT_ID` and `AZURE_FEDERATED_TOKEN_FILE` | On GitHub the step requests the job's OIDC token for the cloud's audience and writes the variables to `$GITHUB_ENV`, so the later steps (the report too) have them. On GitLab the job writes its `id_tokens` variable to the file, and the variables are the job's own, so `after_script` has them. Forgejo gives a job no token, so there a login is rendered only when `tokenFile` names a token the runner provides; without one, the runner's identity mints. `tokenFile` also moves the file on GitHub (default `$RUNNER_TEMP/yodel--token`) and GitLab (`/tmp/yodel--token`). `environments.` gives one environment's roles a login of their own; a job that holds two environments' roles with different logins for one cloud is refused, since it logs in once per cloud. The job's image needs the cloud's command line (`aws`, `gcloud`, `az`). `yodel ci` only renders these steps; nothing calls a cloud until the job runs. Trust the writer's cloud role from the apply job only: on GitHub by the subject `repo:/:environment:`, on GitLab by `ref:main` and the environment. ### yodel config check `yodel config check [...]` prints each environment's settings and where each came from: the server, the role this process connects as, each role's user and password variable and whether it is set here (or the token source), the history's database, the topology and the lock settings. It never prints a password. `--write-probe` tries a write with the credentials this process holds and exits 4 when the server lets it through: a `CREATE TABLE` in the environment's first schema inside a transaction that is rolled back (Postgres), or a `Memory` table in its first database, dropped at once (ClickHouse). Only a refusal for want of privilege counts as "cannot write"; a probe that cannot tell is an error. Every pull request plan job `yodel ci` renders runs it before the plan, with each reader the job holds (`yodel config check prod dev --write-probe`), so a pull request job that holds a writer fails. The pull request job that records the plans a `pr-review` wave's review approves (`yodel-apply-plans.yml`, GitLab's `yodel-apply-record-plans`) runs it too, before `chant run wave --record-plans`, with the reader of every environment a wave applies. ### Least privilege on each forge | Job | Holds | |---|---| | lint (pull request) | no database credentials: the replay runs on a throwaway server the job starts | | plan-`` (pull request) | ``'s reader, and the reader of the environment its wave requires | | watch-`` (schedule) | ``'s reader | | wave `` (push to main) | ``'s writer (`YODEL_CREDENTIALS=writer:`), and the reader of the environment it requires | The pipelines map only those, but a pull request can change a workflow, so where a secret is readable decides what a pull request could take. [Setting up each forge](/sql-yodeler/forges/) says where the readers' and the writer's secrets go on GitHub, GitLab and Forgejo so that only the wave job can read the writer's, and which tokens the jobs use. The database grants are the other half: the reader with SELECT (and on Postgres `default_transaction_read_only = on`; on ClickHouse `readonly = 1`), the writer with what the migrations need on the environment's schema or database and the history's. The write probe proves the first half on every pull request. ## yodel.config.ts Optional. Keyed by the profile names in `chant.config.ts`: ```ts import { defineConfig } from "@intentius/sql-yodeler"; export default defineConfig({ environments: { dev: { topology: "single", history: { database: "yodeler" } }, prod: { topology: "cluster:main", history: { database: "yodeler" }, lockTtl: 1800, lockWait: 600 }, }, lint: { rules: { "ch-mutation": "warning" } }, }); ``` `import type { YodelConfig } from "@intentius/sql-yodeler"` with `satisfies YodelConfig`, as the templates and examples do, types it the same way. | Setting | 1. yodel.config.ts | 2. chant.config.ts | 3. variable | Default | |---|---|---|---|---| | topology (ClickHouse) | `environments..topology` | `sql.profiles..topology` | `YODEL_TOPOLOGY` | `single` | | which role a process connects as | | `credentials()` in chant.config.ts | `YODEL_CREDENTIALS`: `writer` or `writer:` connects as the writer. Only an environment's wave job sets it | the reader | | the history's database (ClickHouse) or schema (Postgres) | `environments..history.database` | | `YODEL_HISTORY_DATABASE` | `yodeler` | | the ClickHouse apply lock's TTL, in seconds | `environments..lockTtl` | | `YODEL_LOCK_TTL` | 900 | | how long an apply waits for a lock another apply holds | `environments..lockWait` (seconds) | | `YODEL_LOCK_WAIT` (a duration: `90`, `30s`, `10m`, `1h`) | 0: exit 5 at once | | access control: Postgres row-level security, policies, roles, grants; ClickHouse users, roles, row policies, grants ([Access control](/sql-yodeler/access/)) | `environments..access` | `sql.profiles..access` | | off | | lint rule levels | `lint.rules` | | | each rule's own | | the apply pipeline's waves and their gates | `waves` | | | none: every environment waits for an approval | | a tenant set: one environment over many databases or schemas | `environments..tenants` | | | none | | checks and commands run before and after an apply | `environments..steps` (read at the base commit) | | | none | | which approvals count: a wave's (`ledger`, `pr-review`, `sealed`), and the migrations Op's (`ledger`, `sealed`) | `waves[].approval`, `environments..approval` (read at the base commit) | | | `ledger`; the migrations Op takes `sealed` from its wave | `waves` lists the environments in the order the apply pipeline applies them, each with a gate policy: `[{ env: "dev", gate: "never" }, { env: "prod", gate: "on-destructive" }]`. An environment it does not name, or a wave without `gate`, waits for an approval whenever it has a migration to apply (`always`). The pipeline reads a wave's gate from the commit a change merges onto, not from the change; [Approval](/sql-yodeler/approval/#waves-a-gate-per-environment) has the rest. A wave applies only migrations the wave before it has applied; `requires: ""` names another environment, `requires: false` none ([Promotion](/sql-yodeler/approval/#promotion-what-the-wave-before-has-applied)). A wave's `approval` says which approvals of its plan count: any approval (`yodel approve`; `ledger`, the default), also the pull request's review (`pr-review`), or only a sealed one (`sealed`); [Approval modes](/sql-yodeler/approval/#approval-modes-which-approvals-count) has the rest. `tenants` makes an environment a tenant set: the same migrations applied to many ClickHouse databases or Postgres schemas, each with its own history. It is a list of names, `{ file: "tenants.txt" }` (one per line, or a JSON array), or `{ query: "SELECT ..." }`, read-only, run on the environment's server each time the wave runs (the rendered pipeline does not list those tenants). A wave over a tenant set can be split with `shares`. [Tenant sets](/sql-yodeler/approval/#tenant-sets-one-wave-over-many-databases-or-schemas) has the rest. `ci` is what `yodel ci` renders the project's pipelines for: `forges`, the forges (default `["github", "gitlab", "forgejo"]`; a forge left out gets no files), and `jobs`, a module relative to the project that declares the project's own jobs with chant's lexicons: `export const actions = { ... }` (the github lexicon's `Job`, for GitHub and Forgejo) and `export const gitlab = { ... }` (the gitlab lexicon's `Job`). `yodel ci` adds them to the pull request pipeline after its own jobs (a GitLab job's stage goes after `review`), so they are kept every time the pipelines are rendered again; a name yodel's jobs use is refused. `image`, when set, is the CI image every job runs in, pinned by its digest (`ghcr.io/intentius/sql-yodeler:@sha256:`; a tag alone is refused): it carries node, yodel, chant and the lexicons, so no job runs `npm ci`. `apply: false` renders no apply pipeline and no `yodel-waves.json`, so CI holds readers only; it is refused with a `pr-review` wave or `resume` ([Lint and plan on pull requests only](/sql-yodeler/lint-and-plan/)). See [The pipelines](/sql-yodeler/workflows/#the-pipelines-yodel-ci). `sources` names DDL sources, the tables an ORM defines read from the DDL it prints; see [Schema from an ORM](/sql-yodeler/orm/). The first one set wins. On ClickHouse, `yodel status ` and `yodel plan ` print the topology and where it came from (`topology single (default)`); Postgres has no topology, and they print none. The declarative path plans and applies through chant, which reads only `sql.profiles..topology`. So a topology set in `yodel.config.ts` alone is refused there unless it is `single`, which is what chant renders when its profile sets none: ```console $ yodel plan dev yodel plan: yodel.config.ts sets environments.dev.topology to cluster:main, but the declarative path plans and applies through chant, which reads sql.profiles.dev.topology (unset, so a single node). Set sql.profiles.dev.topology to the same in chant.config.ts, or leave the topology to chant.config.ts. ``` `access` is the same: the declarative apply and `yodel drift` run chant, which reads `sql.profiles..access`, so on the declarative path an `access` in `yodel.config.ts` that the profile does not say is refused. yodel's own commands (`yodel plan`, `yodel new`, `yodel init --from`, the versioned apply) use `yodel.config.ts`'s. On ClickHouse, chant's apply has no access setting and makes every access declaration it is given, so the declarative path is refused instead where the environment does not manage access and `src/` declares a user, role, row policy or grant ([Access control](/sql-yodeler/access/#turn-it-on-per-environment)). ## Environment variables | Variable | Read by | Meaning | |---|---|---| | `YODEL_TOPOLOGY` | the versioned commands on ClickHouse (`apply`, `plan`, `status`, `init`, `repair`, `lint --replay`) | the topology, when neither config sets one; the declarative path does not read it | | `YODEL_HISTORY_DATABASE` | the versioned commands | the history's database or schema, when `yodel.config.ts` sets none | | `YODEL_LOCK_TTL` | `yodel apply`, `yodel init`, `yodel repair` on ClickHouse | the lock's TTL in seconds | | `YODEL_LOCK_WAIT` | `yodel apply` | how long to wait for a lock another apply holds, as `--lock-wait` (`10m`); `yodel apply` hands its `--lock-wait` to the Op's Apply step this way | | `YODEL_STAND_DOWN` | `yodel apply` | `1`: as `--stand-down`, apply nothing and exit 0 when the run's branch has a newer commit on `origin` | | `YODEL_LOCK_RETRIES` | `yodel apply` on Postgres | how many more times a statement that hit `lock_timeout` is tried (default 5, the pause doubling from 250 ms) | | `GITHUB_TOKEN`, `GITLAB_TOKEN`, `FORGEJO_TOKEN` | `yodel plan --comment`, `yodel drift --issue` | the forge token; see [Approval](/sql-yodeler/approval/) and [Drift](/sql-yodeler/drift/#the-tracking-issue) | `YODEL_APPROVED_PLAN` and `YODEL_ALLOW_OUT_OF_ORDER` are set by `yodel apply` for the migrations Op's own steps; you do not set them. ## Proven by The scenario claims below run what this page describes against a real server, once plain (it passes) and once with the behaviour broken (the claim catches it). [Claims status](/sql-yodeler/claims/) lists every claim. | Claim | What it says | Plain, broken | Last run | |---|---|---|---| | `topology` | ClickHouse: one migrations directory applies to a single node and to a Replicated database, and one to a sharded cluster with drift read there, each rendered for its topology | ClickHouse: pass, caught | `c6f58a4`, 2026-10-10 | | `declarative` | the declarative path: yodel plan shows the change against the live server and yodel apply makes it behind the plan-bound gate, for src/ declarations (on Postgres, functions, procedures and triggers, and access control: a role, a policy and grants, with a grant made by hand revoked; on ClickHouse, a dictionary, a function, and access control: a role, a user, a row policy and grants, with a grant made by hand revoked) and for an ORM's exported DDL | ClickHouse: pass, caught; Postgres: pass, caught | `c6f58a4`, 2026-10-10 | | `approval` | a pending migration applies only after chant approve of its plan digest, and the history records that digest; a failing pre-migration check refuses it, and so does a policy rule, read at the base commit, unless an override is recorded for that digest, for a yodel revert as for an apply; the audit log derived from the history and the ledger accounts for every approval and apply, and names one removed; a reader and a writer that are one user are refused, and the reader, its password a minted token, cannot write; an approval is refused once the migration changed after it, and nothing is applied | ClickHouse: pass, caught; Postgres: pass, caught | `c6f58a4`, 2026-10-10 | --- # Lint Source: https://intentius.io/sql-yodeler/lint/ ## Optional: hand this page to your coding agent ```text Fix the `yodel lint` findings on this branch, following https://intentius.io/sql-yodeler/lint/. Change a migration that is applied nowhere, or write the change another way; add a `-- yodel:allow` line only with a reason I give you, and record the checksum with `npx yodel lint --update-checksum `. Never run `yodel apply` against a shared environment, never run `chant approve`, `yodel approve` or `yodel override`, never edit the `chant/lifecycle` branch or `.chant/allowed_signers`, never merge; approvals and applies belong to people. ``` `yodel lint` checks the migrations directory against every rule, offline: no server and no dev database. It runs in CI on every pull request; the starter templates' pull request job runs it. Every rule is free. Exit codes: 0 no error (warnings and silenced findings do not fail), 3 at least one error, 1 lint could not run; with `--replay`, 4 when the target already holds what the history names and `--reset` was not given. `--json` prints the findings as JSON. The output quoted here is from the examples' runs. ## The rules `yodel lint --rules` lists them with their levels (here in the ClickHouse example, which sets none): ```console $ npx yodel lint --rules fork error two migrations follow the same parent; rebase one onto the other (yodel rebase) directory error the migrations directory is not one readable chain: a bad name or file, a missing parent, a cycle, another dialect checksum error a migration's files no longer hash to its checksum: an edit after it was written, or (with --env) after it was applied unreadable-object error an object in a recorded schema, or a step, the sql lexicon cannot read or classify; never skipped checkpoint error a checkpoint whose recorded schema is no longer the one it was written from, its parent's, or that holds a step other than a statement destructive error, silenceable a statement or Op step removes data that cannot be recovered (a table or column drop) pg-rewrite error, silenceable, postgres Postgres: rewrites or scans the whole table under ACCESS EXCLUSIVE, blocking reads and writes while it runs pg-lock error, silenceable, postgres Postgres: holds a lock that blocks writes for a whole scan or index build (an index without CONCURRENTLY, a constraint validated in place) pg-signature error, silenceable, postgres Postgres: a function or procedure dropped and created with another signature or result; a caller written for the old one fails pg-routine-in-use error, postgres Postgres: a function dropped while a trigger the migration's schema declares still executes it; the DROP fails pg-access error, silenceable, postgres Postgres: takes access away: a privilege revoked, a policy dropped, or row-level security turned off or no longer forced on the table's owner pg-rls-not-forced warning, silenceable, postgres Postgres: a table's row-level security is enabled but not forced, so the table's owner (often the role that applies) bypasses its policies data-dependent warning, silenceable, postgres Postgres: a statement that can fail on the rows already there (a unique index over duplicates, SET NOT NULL over NULLs, a check or foreign key some row breaks); yodel apply runs its pre-check first and refuses when it counts any naming error, silenceable a table, column, index or constraint the migration adds breaks yodel.config.ts lint.naming ch-mutation error, silenceable, clickhouse ClickHouse: starts a mutation that rewrites existing parts in the background, with no rollback ch-rebuild error, silenceable, clickhouse ClickHouse: a rebuild, every row copied into a new table and swapped in (ClickHouseRebuildOp) ch-access error, silenceable, clickhouse ClickHouse: takes access away: a privilege or role revoked from a user or role (SQLCH274) refused-step error a step yodel apply refuses to run: a manual step, or an Op step it does not run migration-sql error migration.sql does not hold a statement as migration.json records it; apply runs migration.json's silence error a yodel:allow line that is malformed, names a rule no statement can silence, has no reason, or silences nothing replay error with --replay : the history, replayed into a fresh database, does not give each migration's recorded schema SQL101 the check's level, silenceable schema check: Two exports declare the same object SQLCH101 the check's level, silenceable, clickhouse schema check: A ClickHouse object names an engine the pinned server does not have SQLCH102 the check's level, silenceable, clickhouse schema check: A PRIMARY KEY is not a prefix of ORDER BY SQLCH103 the check's level, silenceable, clickhouse schema check: An engine's version, sign or is_deleted column has an unsupported type SQLCH104 the check's level, silenceable, clickhouse schema check: A MergeTree engine argument names a column the table does not declare SQLCH105 the check's level, silenceable, clickhouse schema check: A table key clause names a column the table does not declare SQLCH106 the check's level, silenceable, clickhouse schema check: A skip index expression names a column the table does not declare SQLCH107 the check's level, silenceable, clickhouse schema check: A TTL expression is built on a column that is not a Date or DateTime SQLCH108 the check's level, silenceable, clickhouse schema check: CREATE OR REPLACE TABLE in a database that is not Atomic SQLCH109 the check's level, silenceable, clickhouse schema check: A materialized view selects * SQLCH110 the check's level, silenceable, clickhouse schema check: A materialized view writes a column its TO target does not declare SQLCH111 the check's level, silenceable, clickhouse schema check: A String column limited to a few values is not LowCardinality SQLCH112 the check's level, silenceable, clickhouse schema check: A PARTITION BY expression is finer than a day SQLCH113 the check's level, silenceable, clickhouse schema check: A MergeTree table declares no sort key SQLCH114 the check's level, silenceable, clickhouse schema check: A column codec is not one the pinned server has SQLCH115 the check's level, silenceable, clickhouse schema check: A secret- or PII-named column carries no comment, TTL or encryption SQLCH116 the check's level, silenceable, clickhouse schema check: A view is SQL SECURITY DEFINER with no DEFINER SQLCH117 the check's level, silenceable, clickhouse schema check: A MergeTree setting is obsolete at the pinned server SQLCH118 the check's level, silenceable, clickhouse schema check: A MergeTree setting is not one the pinned server has SQLCH119 the check's level, silenceable, clickhouse schema check: An object uses a deprecated or experimental engine SQLCH120 the check's level, silenceable, clickhouse schema check: A MergeTree engine uses the deprecated positional arguments SQLCH121 the check's level, silenceable, clickhouse schema check: A column type names a family the pinned server does not have SQLCH122 the check's level, silenceable, clickhouse schema check: A column type's parameters do not fit its family SQLCH123 the check's level, silenceable, clickhouse schema check: A Nullable wraps a type ClickHouse does not allow inside Nullable SQLCH124 the check's level, silenceable, clickhouse schema check: A table declares a column twice SQLCH125 the check's level, silenceable, clickhouse schema check: A codec parameter is outside what the codec takes SQLCH126 the check's level, silenceable, clickhouse schema check: A GRANT column list names a column the table does not declare SQLCH127 the check's level, silenceable, clickhouse schema check: An expression calls a function the pinned server does not have ``` `fork`, `directory`, `checksum`, `unreadable-object` and `checkpoint` are integrity rules: always errors. `unreadable-object` means an object in a recorded schema, or a step, that the `sql` lexicon cannot read or classify; it is never skipped. `destructive`, `pg-rewrite`, `pg-lock`, `ch-mutation` and `ch-rebuild` come from the classifier's class of each statement or step, and can be silenced one statement at a time. The `pg-` and `ch-` rules apply to their own dialect. Two Postgres rules are about access control ([Access control](/sql-yodeler/access/)). `pg-access` flags a statement that takes access away: a privilege revoked (SQLPG297), a policy dropped (SQLPG292), row-level security turned off or no longer forced (SQLPG293). A revoke on an object the same migration creates is left out, since nobody held the privilege. A wave with gate `on-destructive` waits on it. `pg-rls-not-forced`, a warning, flags a migration that enables a table's row-level security while its recorded schema does not force it, so the table's owner is not held to its policies. On ClickHouse, `ch-access` flags a privilege or role revoked from a user or role (SQLCH274). ClickHouse takes no `REVOKE` declaration: the grant declarations naming a grantee are everything it holds, so a revoke always takes away something no declaration grants any more. A wave with gate `on-destructive` waits on it too. Two Postgres rules read a step with the rest of its migration (#44). `pg-signature` flags a function or procedure dropped and created with another signature: SQLPG281 (the result type, an output parameter, an input parameter's name or a removed default, which `CREATE OR REPLACE` refuses, or other parameter types between two builds), or a routine dropped and one of the same name with other parameter types created in the same migration. A caller written for the old signature fails once it runs; it can be silenced for the statement. `pg-routine-in-use` flags a function dropped while a trigger in the migration's recorded schema still executes it, which happens when the trigger names the function in its text rather than through its declaration. The `DROP` would fail on the server, so it cannot be silenced: declare the function again, or drop the trigger in the same migration. Findings from the Postgres example. A CHECK constraint added to a table that holds rows, which `yodel new` writes `NOT VALID` and then validates, so the validation carries a pre-check ([Statements that can fail on the data](#statements-that-can-fail-on-the-data)); the baseline's warning is [`SQLPG112`](#schema-checks), a column that looks like personal data with no comment: ```console $ npx yodel lint 20261010T1723-baseline warning SQLPG112: step 2: orders.customer_email looks like a secret or personal data and neither it nor shop.orders has a COMMENT; say what it holds and how it is protected 20261010T1723-coupons warning data-dependent: step 1: SQLPG217 Add a constraint NOT VALID on shop.orders: can fail on the rows already there (rows that fail the check orders_amount_positive (amount > 0)); yodel apply runs this pre-check before the migration's first statement and refuses when it is not 0: SELECT count(*) AS n FROM shop.orders WHERE NOT (amount > 0) 2 migrations: 0 errors, 2 warnings, 0 silenced. ``` an index built without `CONCURRENTLY`: ```console $ npx yodel lint 20261010T1723-baseline warning SQLPG112: step 2: orders.customer_email looks like a secret or personal data and neither it nor shop.orders has a COMMENT; say what it holds and how it is protected 20261010T1723-coupons warning data-dependent: step 1: SQLPG217 Add a constraint NOT VALID on shop.orders: can fail on the rows already there (rows that fail the check orders_amount_positive (amount > 0)); yodel apply runs this pre-check before the migration's first statement and refuses when it is not 0: SELECT count(*) AS n FROM shop.orders WHERE NOT (amount > 0) 20261010T1724-drop-coupon silenced destructive: step 0: SQLPG204 Drop a column on shop.orders: removes data that cannot be recovered reason: no order ever had a coupon; checked on dev 20261010T1724-rename-email warning SQLPG112: step 0: orders.email looks like a secret or personal data and neither it nor shop.orders has a COMMENT; say what it holds and how it is protected 20261010T1725-index-placed-at error pg-lock: step 0: SQLPG241 Create an index without CONCURRENTLY on shop.orders_placed_at_idx: builds an index without CONCURRENTLY: SHARE on the table, blocking every write until the build ends; declare the index CONCURRENTLY A statement rule can be silenced for one statement, with a reason the history records when it applies: a line `-- yodel:allow ` above the statement in migration.sql 6 migrations: 1 error, 3 warnings, 1 silenced. [exit 3] ``` and a column dropped: ```console $ npx yodel lint 20261010T1723-baseline warning SQLPG112: step 2: orders.customer_email looks like a secret or personal data and neither it nor shop.orders has a COMMENT; say what it holds and how it is protected 20261010T1723-coupons warning data-dependent: step 1: SQLPG217 Add a constraint NOT VALID on shop.orders: can fail on the rows already there (rows that fail the check orders_amount_positive (amount > 0)); yodel apply runs this pre-check before the migration's first statement and refuses when it is not 0: SELECT count(*) AS n FROM shop.orders WHERE NOT (amount > 0) 20261010T1724-drop-coupon error destructive: step 0: SQLPG204 Drop a column on shop.orders: removes data that cannot be recovered 20261010T1724-rename-email warning SQLPG112: step 0: orders.email looks like a secret or personal data and neither it nor shop.orders has a COMMENT; say what it holds and how it is protected A statement rule can be silenced for one statement, with a reason the history records when it applies: a line `-- yodel:allow ` above the statement in migration.sql 5 migrations: 1 error, 3 warnings, 0 silenced. [exit 3] ``` ## Rule levels `yodel.config.ts` sets a rule to `"error"`, `"warning"` or `"off"`: ```ts export default defineConfig({ lint: { rules: { "ch-mutation": "warning", "pg-lock": "off" } }, }); ``` The integrity rules cannot be lowered. A schema check's id is a rule too (`"SQLPG104": "off"`), and `lint.naming` sets the [naming rules](#naming-rules). ## Schema checks The `sql` lexicon's schema checks, the ones `chant lint` runs over a build, run inside `yodel lint` on each migration's recorded schema, so a project has one lint command and one configuration. Each check is a rule named by its id: `SQLPG101` (a table with no primary key), `SQLPG102` (a foreign key whose columns have no index), `SQLPG104` (a timestamp without time zone), `SQLCH101` (an engine the pinned server does not have), `SQLCH113` (a MergeTree table with no sort key), and the rest that `yodel lint --rules` lists for the project's dialect. A finding is reported once, in the migration whose recorded schema first has it: the parent's recorded schema is checked too, and what it already had is left out. It is placed at the migration's first step on the object it names, so a `-- yodel:allow SQLPG101 ` line above that step silences it. In the Postgres example, the first migration's `customer_email` column and the later rename's `email` are each reported where they appear: ```text 20261010T1723-baseline warning SQLPG112: step 2: orders.customer_email looks like a secret or personal data and neither it nor shop.orders has a COMMENT; say what it holds and how it is protected ``` A check's level is, in order: `lint.rules` in `yodel.config.ts`; else `lint.rules` in `chant.config.ts`, the level `chant lint` uses (`info` counts as a warning); else the level the check gives the finding. `"off"` in either file turns it off. The checks read the objects as `chant build` writes them, with their columns, keys and engine besides the DDL; that is what `yodel new` records. ## Naming rules `lint.naming` in `yodel.config.ts` sets a rule per kind of object: `table`, `column`, `index`, `constraint`. A rule is a preset, `snake_case`, `camelCase` or `PascalCase`, or a regular expression the whole name must match: ```ts export default defineConfig({ lint: { naming: { table: "snake_case", column: "snake_case", index: "[a-z_]+_idx", constraint: "[a-z_]+_(pkey|key|fkey|check)" }, }, }); ``` Rule `naming` (an error by default) flags each name a migration adds that breaks its kind's rule: a table, column, index or named constraint in its recorded schema that its parent's does not have. A name already there is not reported again, so a rule added to a project with migrations does not fail what was written before it; the first migration's names are all new. The names are the server's: Postgres folds an unquoted name to lower case, so `CREATE TABLE OrderItems` is `orderitems`. A constraint the declaration does not name is named by the server and is not checked. A ClickHouse table's skip indexes and constraints are its indexes and constraints. The finding is at the migration's first step on the object, and `-- yodel:allow naming ` above it silences it. ## Statements that can fail on the data Some Postgres statements succeed or fail by what is in the table: a unique index or constraint over duplicate values, `SET NOT NULL` over NULLs, a CHECK some row breaks, a foreign key with orphan rows. chant marks each such statement with a pre-check, a query counting the rows it would fail on, and `migration.json` records it on the step (so it is in the checksum and the plan digest). `migration.sql` shows it above the statement: ```sql -- ordersStatusKey (yodel_34_lint.orders_status_key): SQLPG240 concurrently, outside a transaction -- pre-check, must return 0 (values of (status) held by more than one row, which a unique index fails on): SELECT (SELECT count(*) FROM (SELECT 1 FROM yodel_34_lint.orders WHERE status IS NOT NULL GROUP BY status HAVING count(*) > 1) AS duplicates) AS n CREATE UNIQUE INDEX CONCURRENTLY orders_status_key ON yodel_34_lint.orders (status); ``` Rule `data-dependent`, a warning, flags each one with its pre-check, so a reviewer sees what the statement depends on. `yodel apply` runs the pre-checks with the migration's own [pre-migration checks](/sql-yodeler/migrations/#pre-migration-checks): before the gate, and again under the lock just before the migration's first statement. A pre-check that returns more than 0 refuses the apply with exit 4, naming the statement and the count, and nothing of the migration is sent: ```text $ yodel apply lint yodel apply: refused: a pre-migration check of 20261010T1311-unique-status failed before the gate. Nothing of 20261010T1311-unique-status was applied: step 0, CREATE UNIQUE INDEX CONCURRENTLY orders_status_key ON yodel_34_lint.orders (status), would fail: 2 values of (status) held by more than one row, which a unique index fails on (pre-check precheck-0: SELECT (SELECT count(*) FROM (SELECT 1 FROM yodel_34_lint.orders WHERE status IS NOT NULL GROUP BY status HAVING count(*) > 1) AS duplicates) AS n) Fix the data, then run yodel apply again; if the check is wrong, edit it in migration.json and run yodel lint --update-checksum 20261010T1311-unique-status (the plan digest moves, so the plan is approved again). A statement's pre-check is chant's count of the rows the statement fails on: fix those rows, or change the declaration. [exit 4] ``` Both are from the `lint` claim's run, where `orders` holds two rows of each of two statuses and the declaration gains `CREATE UNIQUE INDEX CONCURRENTLY orders_status_key ON orders (status)`. `yodel lint` reports the same statement: ```text 20261010T1311-unique-status warning data-dependent: step 0: SQLPG240 Create an index CONCURRENTLY on yodel_34_lint.orders_status_key: can fail on the rows already there (values of (status) held by more than one row, which a unique index fails on); yodel apply runs this pre-check before the migration's first statement and refuses when it is not 0: SELECT (SELECT count(*) FROM (SELECT 1 FROM yodel_34_lint.orders WHERE status IS NOT NULL GROUP BY status HAVING count(*) > 1) AS duplicates) AS n ``` Fix the rows, or change the declaration, and apply again. The plan and the pull request comment list the pre-checks with the migration's checks, and `yodel new` prints them when it writes the migration. A pre-check that reads a table or column an earlier statement of the same migration adds cannot run before the first statement; it is passed, and if the statement then fails on the data, the migration stops there and resumes at it on the next run. Migrations `yodel new` writes use chant's lock-safe forms. On a table that exists, `SET NOT NULL` is a CHECK (`col IS NOT NULL`) added `NOT VALID` (SQLPG217, no rows read), validated (SQLPG220, under SHARE UPDATE EXCLUSIVE, so reads and writes go on), then `SET NOT NULL`, which the valid check proves without a scan, and the check dropped. A CHECK or a foreign key is added `NOT VALID`, then validated. Each statement runs in a transaction of its own, so no lock that blocks reads or writes is held across a scan. The plan shows each step's rule and class (`metadata`, `validate`, `concurrently`, `rewrite`), and the first statement of each sequence carries the pre-check. ## Silencing a finding A line above a statement in its `migration.sql` silences one of the classifier rules, `data-dependent`, `naming` or a schema check for that statement: ```sql -- yodel:allow ``` The reason is required. For an Op step, the line goes above the step's comment. From the Postgres example: ```sql -- yodel migration 20261010T1724-drop-coupon -- parent: 20261010T1724-rename-email -- orders (shop.orders): SQLPG204 metadata, destructive -- yodel:allow destructive no order ever had a coupon; checked on dev ALTER TABLE shop.orders DROP COLUMN coupon; ``` The line is part of `migration.sql`, so it changes the migration's checksum, and lint says so. In the ClickHouse example, after silencing the rebuild: ```console $ npx yodel lint 20261010T1722-events-by-id error checksum: 20261010T1722-events-by-id: its files hash to sha256:957ddc1e1c31ff222c7fa567e8bcb5bc6cedef03f621d0263da0913a7c4f953c, but migration.json records sha256:e3c2c97aea554b9df6c0ad70147403180c045dc321f545c0cef0dee9bb7358c4; if it is not applied in any environment, record its new checksum with yodel lint --update-checksum 20261010T1722-events-by-id; if it is, undo the edit, or record it with yodel repair 20261010T1722-events-by-id --reason ... silenced ch-rebuild: step 0: ClickHouseRebuildOp rebuilds shop.events: every row is copied into a new table and swapped in, for SQLCH220 Change the sorting key (orderBy) reason: 600 rows; the copy takes seconds 3 migrations: 1 error, 0 warnings, 1 silenced. [exit 3] ``` For a migration not applied anywhere yet, `--update-checksum ` (repeatable) writes the new checksum into its `migration.json` before linting: ```console $ npx yodel lint --update-checksum 20261010T1724-drop-coupon 20261010T1724-drop-coupon: checksum sha256:a249036ee571dacb3190a254750b79e55d6920c82aeed07f779e55e911d0973b is now sha256:326cf9df2b47950bfe361be91935ad57382906f28f8f2555dcf930b030cc6a4e 20261010T1723-baseline warning SQLPG112: step 2: orders.customer_email looks like a secret or personal data and neither it nor shop.orders has a COMMENT; say what it holds and how it is protected 20261010T1723-coupons warning data-dependent: step 1: SQLPG217 Add a constraint NOT VALID on shop.orders: can fail on the rows already there (rows that fail the check orders_amount_positive (amount > 0)); yodel apply runs this pre-check before the migration's first statement and refuses when it is not 0: SELECT count(*) AS n FROM shop.orders WHERE NOT (amount > 0) 20261010T1724-drop-coupon silenced destructive: step 0: SQLPG204 Drop a column on shop.orders: removes data that cannot be recovered reason: no order ever had a coupon; checked on dev 20261010T1724-rename-email warning SQLPG112: step 0: orders.email looks like a secret or personal data and neither it nor shop.orders has a COMMENT; say what it holds and how it is protected 5 migrations: 0 errors, 3 warnings, 1 silenced. ``` With `--env ` it refuses a migration that environment has applied. An applied migration's edit is recorded with `yodel repair` instead ([Migrations](/sql-yodeler/migrations/#yodel-repair)). `yodel apply` records each silence (step, rule, reason) in every history row the migration writes, in the `silences` column: ```sql SELECT kind, statement_index, status, silences FROM yodeler.history WHERE migration_id LIKE '%drop-coupon' ORDER BY seq ``` ``` kind statement_index status silences migration -1 started [{"step":0,"rule":"destructive","reason":"no order ever had a coupon; checked on dev"}] statement 0 started [{"step":0,"rule":"destructive","reason":"no order ever had a coupon; checked on dev"}] statement 0 succeeded [{"step":0,"rule":"destructive","reason":"no order ever had a coupon; checked on dev"}] migration -1 succeeded [{"step":0,"rule":"destructive","reason":"no order ever had a coupon; checked on dev"}] ``` A `yodel:allow` line that is malformed, names a rule that cannot be silenced, has no reason, or silences nothing is itself an error (rule `silence`). `yodel rebase` keeps each `yodel:allow` line above the same statement in the migration.sql it writes again, and the new checksum covers it, so lint passes after a rebase and apply records the same step, rule and reason. When the statement a line silenced is gone from the rebased migration, or changed (the other branch already made that change, say), the rebase refuses, names the line, and writes nothing ([Migrations](/sql-yodeler/migrations/#forks-and-yodel-rebase)). ## Against an environment's history: --env `yodel lint --env ` also reads that environment's history. A `checksum` finding is then an applied migration whose files changed since it ran (an error) or a pending migration's edit (a warning). It needs the server; a server that does not answer exits 1. ## The replay check: --replay `yodel lint --replay ` also replays every migration, in order, into ``'s server and compares the schema each one leaves with the schema it recorded. The first that differs fails the run (rule `replay`, exit 3) and is named. This is what keeps the recorded schemas, which `yodel new` diffs against, true to what the statements do. `` should be a throwaway server: the emulator (`npx yodel emulator up`), or in CI a service container, as the starter templates' `replay` profile is. The databases or schemas the history names must not exist there yet; otherwise it runs nothing and exits 4: ```console $ npx yodel lint --replay replay yodel lint: refused: replay and dev are the same server (127.0.0.1:8123) and both name shop; the replay would drop dev's database, with or without --reset. Point replay at a throwaway server of its own (a second emulator on another port), or, to empty dev too, add --reset-shared dev. Nothing was run. [exit 4] ``` A checkpoint ([Migrations](/sql-yodeler/migrations/#checkpoints-yodel-checkpoint)) is replayed on its own: after the chain, the database is emptied again and the checkpoint's statements run into it, and the result must be its recorded schema. `--reset` drops them first. Never point it at a server whose data matters. `--keep` leaves the replayed database in place to look at. Rebuild and `PostgresMigrationOp` steps run in the replay; backfills are skipped, since the replayed database has no data. The end of the ClickHouse example's run: ```console $ npx yodel lint --replay replay --reset --reset-shared dev ... Replay of 5 migrations (clickhouse) into http://127.0.0.1:8123 (sql.profiles.replay) databases: shop (emptied first) ok 20261010T1722-baseline (2 statements) ok 20261010T1722-add-country (1 statement) ok 20261010T1722-events-by-id (0 statements; ran step 0: ClickHouseRebuildOp for events (shop.events) (swapped)) ok 20261010T1722-fill-country (0 statements; skipped step 0: backfill backfill.ts (moves data; the replayed database has none)) ok 20261010T1723-add-source (1 statement) Every migration replays to its recorded schema. 20261010T1722-events-by-id silenced ch-rebuild: step 0: ClickHouseRebuildOp rebuilds shop.events: every row is copied into a new table and swapped in, for SQLCH220 Change the sorting key (orderBy) reason: 600 rows; the copy takes seconds 5 migrations: 0 errors, 0 warnings, 1 silenced. ``` ## Findings on the pull request: --format `yodel lint --format github` prints the usual report, then one workflow command per finding, such as `::error file=migrations//migration.sql,line=3,title=yodel lint destructive::...`. GitHub Actions and Forgejo Actions read those lines and show each finding on its migration file in the pull request, on the statement's line when the rule knows it. A fork is shown on the newer of the two migrations, the one to rebase. A warning is a `::warning`, and a silenced finding a `::notice` that carries its reason. `yodel lint --format gitlab` prints the same report and writes GitLab's code quality report to `gl-code-quality-report.json` (or the file `--output` names), even when there are no findings. An error is `major`, a warning `minor` and a silenced finding `info`. File paths are relative to the forge's checkout (`GITHUB_WORKSPACE`, `CI_PROJECT_DIR`), or to the project's directory outside CI. `yodel ci` renders each forge's lint job with its format, and on GitLab keeps the report as the job's `artifacts:reports:codequality`, when the job fails too. `--json` is `--format json`. ## Tests on the replayed database: yodel test The replay check shows each migration gives the schema it recorded. It doesn't show that the data survives a migration, that a backfill fills what it should, or that a view returns the right rows: the replayed database has no data, and the replay skips backfills. `yodel test` runs tests for those, written in `tests/*.test.ts`: ```ts import { defineTests, expectError, expectRows, expectValue } from "@intentius/sql-yodeler"; export default defineTests([ { name: "fill-country fills the country of each event in the months it covers", at: "20261010T1722-events-by-id", seed: ["INSERT INTO shop.events (id, kind, at) VALUES (1, 'view', '2026-06-03 10:00:00')"], expect: [expectRows("SELECT id, country FROM shop.events ORDER BY id", [[1, "FR"]])], }, { name: "orders_amount_positive refuses an order of 0", expect: [expectError("INSERT INTO shop.orders (email, amount) VALUES ('c@example.com', 0)", "orders_amount_positive")], }, ]); ``` Each case runs on a database of its own, built on `--env`'s server (default the `test` profile, else `replay`): 1. the databases (or schemas) the history names are emptied; 2. the migrations up to `at` are replayed the way the replay check replays them: statements, rebuild and `PostgresMigrationOp` steps, no backfills. Without `at`, that is every migration, and the case tests the schema the newest one leaves: a view's rows, a function's result, a constraint; 3. the `seed` statements run; 4. the migrations after `at`, up to `through` (default: the newest), run with every step, backfills included. The receipts of a backfill's batches and of a ClickHouse rebuild's partitions are kept in memory for that case only, so nothing is written to the server's receipts table and no case skips work another case did; 5. each expectation is checked, all of them even after one fails. `expectRows(sql, rows)` wants exactly those rows in that order, `expectValue(sql, value)` the first column of the first row, and `expectError(sql, match?)` a statement that fails, with a message that contains `match` (or matches it, for a pattern). Values compare as text, so `1` and `"1"` are equal and a date compares as its ISO string. The helpers only build plain objects (`{ kind: "rows", sql, rows }`, and so on), so a test file can write those objects itself and import only the `TestCase` type. A checkpoint isn't part of the chain the cases build, as in the replay. Nothing is written to a history. At the end the databases are emptied again, unless you pass `--keep`. `yodel test` never runs on an environment yodel applies to. If the target's history table holds a row, it refuses (exit 4), `--reset` or not. As with the replay check, it also refuses a target that already holds what the history names (unless `--reset`), and a profile that shares its server with another profile naming the same databases (unless `--reset-shared `). The run prints each case as `pass`, `FAIL` (with each expectation that did not hold) or `ERROR` (a migration or seed statement that failed, or an unknown migration id), and exits 3 when any case did not pass. `--json` prints the outcome (`schemas/test.schema.json`), `--junit ` writes it as JUnit XML (one test suite per file), and `--run ` runs only the cases whose names match. The examples have tests: `examples/clickhouse/tests/events.test.ts` covers the backfill and the rebuild with rows in the table, and `examples/postgres/tests/orders.test.ts` covers the column rename with rows, plus a constraint and a default. In a project with test files, `yodel ci` adds `yodel test --env replay --reset --junit yodel-test.xml` to the pull request's lint job, after the replay check on the same throwaway server. ## Proven by The scenario claims below run what this page describes against a real server, once plain (it passes) and once with the behaviour broken (the claim catches it). [Claims status](/sql-yodeler/claims/) lists every claim. | Claim | What it says | Plain, broken | Last run | |---|---|---|---| | `lint` | yodel lint --replay replays the migrations into a fresh database and each gives the schema it recorded; offline, yodel lint fails a migrations directory with a fork (exit 3), naming both migrations; a checkpoint replays alone to its recorded schema, and a fresh environment starts from it; yodel revert undoes the newest migration behind the gate, with a hand-written step for its data step, back to its parent's recorded schema, and refuses a checkpoint or a migration before one; yodel test runs a project's tests on databases replayed from the migrations, a seed meeting the backfill after it, fails a case whose assertion does not hold, and refuses an environment with a history; on Postgres, a unique index over rows with duplicates is refused before the gate by its generated pre-check (exit 4, naming the statement and the count), and yodel lint flags it as data-dependent | ClickHouse: pass, caught; Postgres: pass, caught | `c6f58a4`, 2026-10-10 | | `new` | yodel new writes the next migration offline, with no dev database, and it applies; on Postgres, functions, procedures and triggers too, which lint checks | ClickHouse: pass, caught; Postgres: pass, caught | `c6f58a4`, 2026-10-10 | --- # Topology Source: https://intentius.io/sql-yodeler/topology/ A ClickHouse environment runs on one of four topologies. One migrations directory serves all of them: `yodel new` writes every statement for a single node and records nothing about where it will run, and `yodel apply` renders each statement for the environment's topology as it sends it. (Topology does not apply to Postgres.) | Setting | What it is | What a statement gets | |---|---|---| | `single` | one server, the default | sent as written | | `cluster:` | a cluster in `remote_servers` | `ON CLUSTER `; `MergeTree` engines become `ReplicatedMergeTree` with a Keeper path and replica name | | `replicated`, `replicated:` | `Replicated` databases, which replicate their own DDL | no `ON CLUSTER` inside the database; `ReplicatedMergeTree` with no arguments (the database supplies them); with a cluster named, `CREATE DATABASE` itself goes `ON CLUSTER` | | `cloud` | ClickHouse Cloud | no `ON CLUSTER`; plain `MergeTree`, which Cloud turns into `SharedMergeTree` | A dictionary is rendered like a table: `ON CLUSTER` on a cluster, none inside a `Replicated` database. A function, user, role, row policy or grant belongs to no database, so a `Replicated` database's log does not carry it: it goes `ON CLUSTER` on a cluster, and in `replicated:` too (`CREATE FUNCTION f ON CLUSTER main`, `GRANT ON CLUSTER main SELECT ON ...`); in `replicated` with no cluster named it is sent to the node the apply runs on ([Access control](/sql-yodeler/access/#clickhouse)). The object form, `{ kind: "cluster", cluster: "main", replicaPath: "...", replicaName: "..." }`, sets the Keeper path and replica name of the tables the migrations create (the default path is `/clickhouse/tables/{uuid}/{shard}`, the replica `{replica}`). ## Setting it Per environment, first one set wins: `environments..topology` in `yodel.config.ts`, then `sql.profiles..topology` in `chant.config.ts`, then `YODEL_TOPOLOGY`, then `single`. `yodel status` and `yodel plan` print it with where it came from: ```console $ npx yodel status dev Migrations in dev (clickhouse 26.8.15.10 at 127.0.0.1:8123, history yodeler.history, topology single (default)) ... ``` The declarative path (no `migrations/`) plans and applies through chant, which reads only `sql.profiles..topology`. A topology other than `single` set only in `yodel.config.ts` is refused there (chant renders a single node when its profile sets none); set it in `chant.config.ts` too. `yodel drift` reads through chant as well and refuses the same way, on either path: chant compares the server against the declaration rendered for the profile's topology, so a cluster's `ReplicatedMergeTree` would otherwise show as drift from the declared `MergeTree`. A rebuild step (`ClickHouseRebuildOp`) runs through chant as well, and yodel hands it the environment's topology, so it renders for the same topology as the statements around it. ## The same statements on each topology `examples/walkthrough/topology.ts` runs three statements like the ClickHouse example's through the renderer `yodel apply` calls for each statement, and prints the history's DDL for each topology: ```console $ npx tsx examples/walkthrough/topology.ts -- topology: single CREATE DATABASE shop ENGINE = Atomic; CREATE TABLE shop.events (id UInt64, kind LowCardinality(String), at DateTime) ENGINE = MergeTree PARTITION BY toYYYYMM(at) ORDER BY (kind, at); ALTER TABLE `shop`.`events` ADD COLUMN country LowCardinality(String) DEFAULT '' AFTER `at`; -- the history: CREATE DATABASE IF NOT EXISTS `yodeler` -- its table's engine: MergeTree -- topology: cluster:main CREATE DATABASE shop ON CLUSTER `main` ENGINE = Atomic; CREATE TABLE shop.events ON CLUSTER `main` (id UInt64, kind LowCardinality(String), at DateTime) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{uuid}/{shard}', '{replica}') PARTITION BY toYYYYMM(at) ORDER BY (kind, at); ALTER TABLE `shop`.`events` ON CLUSTER `main` ADD COLUMN country LowCardinality(String) DEFAULT '' AFTER `at`; -- the history: CREATE DATABASE IF NOT EXISTS `yodeler` ON CLUSTER `main` -- its table's engine: ReplicatedMergeTree('/clickhouse/yodeler/yodeler/history', '{shard}-{replica}') -- topology: replicated CREATE DATABASE shop ENGINE = Replicated('/clickhouse/databases/shop', '{shard}', '{replica}'); CREATE TABLE shop.events (id UInt64, kind LowCardinality(String), at DateTime) ENGINE = ReplicatedMergeTree PARTITION BY toYYYYMM(at) ORDER BY (kind, at); ALTER TABLE `shop`.`events` ADD COLUMN country LowCardinality(String) DEFAULT '' AFTER `at`; -- the history: CREATE DATABASE IF NOT EXISTS `yodeler` ENGINE = Replicated('/clickhouse/databases/yodeler', '{shard}', '{replica}') -- its table's engine: ReplicatedMergeTree -- topology: replicated:main CREATE DATABASE shop ON CLUSTER `main` ENGINE = Replicated('/clickhouse/databases/shop', '{shard}', '{replica}'); CREATE TABLE shop.events (id UInt64, kind LowCardinality(String), at DateTime) ENGINE = ReplicatedMergeTree PARTITION BY toYYYYMM(at) ORDER BY (kind, at); ALTER TABLE `shop`.`events` ADD COLUMN country LowCardinality(String) DEFAULT '' AFTER `at`; -- the history: CREATE DATABASE IF NOT EXISTS `yodeler` ON CLUSTER `main` ENGINE = Replicated('/clickhouse/databases/yodeler', '{shard}', '{replica}') -- its table's engine: ReplicatedMergeTree -- topology: cloud CREATE DATABASE shop ENGINE = Atomic; CREATE TABLE shop.events (id UInt64, kind LowCardinality(String), at DateTime) ENGINE = MergeTree PARTITION BY toYYYYMM(at) ORDER BY (kind, at); ALTER TABLE `shop`.`events` ADD COLUMN country LowCardinality(String) DEFAULT '' AFTER `at`; -- the history: CREATE DATABASE IF NOT EXISTS `yodeler` -- its table's engine: MergeTree ``` `test/docs.test.ts` checks that this block is what the script prints. ## The history and the lock The history table follows the topology too, as above: `MergeTree` on a single node and on Cloud, one `ReplicatedMergeTree` copy for the whole cluster (its Keeper path names no shard, so every runner on any host reads the same rows), and `ReplicatedMergeTree` inside a `Replicated` database. A statement row records the hash of the statement as written, not as rendered, so a migration that failed part way resumes the same way on any topology; its `note` says what it was sent for (`topology=single`). The plan digest covers the topology, so an approval given for one topology does not apply on another. The apply lock is a row in a `KeeperMap` table, `.lock`, which needs Keeper and the server setting `keeper_map_path_prefix`. On a cluster the table is created `ON CLUSTER`, every host naming the same Keeper path. A single node without them falls back to a lock file on the runner's machine and says so; this is the emulator, in the ClickHouse example: ``` lock: no KeeperMap on this server (KeeperMap is disabled because 'keeper_map_path_prefix' config is not defined. (BAD_ARGUMENTS) (version 26.8.15.10 (official build))); the lock is a file on this machine, so cross-runner locking needs Keeper (and keeper_map_path_prefix) ``` A cluster, a `Replicated` database and Cloud never fall back: without the `KeeperMap` lock, the apply refuses. [Migrations](/sql-yodeler/migrations/#the-apply-lock) says how to clear a lock. ## What has been run where The examples run on a single node (the emulator). `test/e2e/topology.test.ts` applies one migrations directory to the emulator and to a `Replicated` database on a scratch two-replica cluster with Keeper, and `test/e2e/keeper.test.ts` runs the lock and the replicated history on that cluster (both need Docker and `YODEL_E2E_KEEPER=1`). The `topology` claim (`npm run claims -- topology`, `scenarios/claims/topology.ts`) does the same, and also applies to a sharded cluster: a third server joins the scratch cluster as a second shard (`test/e2e/keeper-cluster.ts` with `{ shards: 2 }`), and an environment with topology `cluster:yodel_sharded` gets versioned migrations, `ON CLUSTER`, with `ReplicatedMergeTree` tables and a `Distributed` table over them. The claim checks the tables on every server, rows written through the `Distributed` table landing half on each shard, the history read from the second shard, a sort-key change applied as a rebuild step that copies and verifies every shard (each keeps its own rows, and the `Distributed` table still reads them all), and `yodel drift`: clean after the applies, then naming a TTL changed on the second shard alone, and a TTL changed and a table dropped `ON CLUSTER`. Drift reads every server of the cluster and says which server a difference was seen on. A rebuild on a cluster of more than one shard needs the `{shard}` macro on every server, distinct per shard; without it the step refuses the cluster. What it does not cover yet: - ClickHouse Cloud is untested beyond rendering: the `cloud` output above comes from the renderer and its unit tests, and nothing in this repository has applied to a Cloud service. ## Proven by The scenario claims below run what this page describes against a real server, once plain (it passes) and once with the behaviour broken (the claim catches it). [Claims status](/sql-yodeler/claims/) lists every claim. | Claim | What it says | Plain, broken | Last run | |---|---|---|---| | `topology` | ClickHouse: one migrations directory applies to a single node and to a Replicated database, and one to a sharded cluster with drift read there, each rendered for its topology | ClickHouse: pass, caught | `c6f58a4`, 2026-10-10 | --- # The commands Source: https://intentius.io/sql-yodeler/cli/ Everything a SQL Yodeler project needs is a `yodel` command, and the starter templates add `just` targets that wrap them. This page says which one does what, then lists `yodel`'s commands, its exit codes and the JSON Schemas of its `--json` output. ## Which command runs what | Command | What it is for | |---|---| | `npx @intentius/sql-yodeler create --clickhouse` (or `--postgres`) | once, to make the project from SQL Yodeler's template, the one released with that yodel ([Your first migration](/sql-yodeler/getting-started/#1-make-the-project)). | | `yodel emulator up` | a local ClickHouse and Postgres in Docker, for development; `yodel emulator status` and `yodel emulator down` check and stop them. | | `yodel` | everything about the schema and the migrations: `plan`, `new`, `lint`, `apply`, `status`, `drift`, `report`, `ci` and the rest of the commands `yodel --help` lists below. Inside a project, run it as `npx yodel`. | | `yodel approve ` | a person's approval of the plan `yodel apply` would run, at a terminal: it shows the plan and asks you to type the environment's name. Every message that asks for an approval prints it with the digest, `yodel approve --plan ` ([Approval](/sql-yodeler/approval/)). | | the template's `just` targets | optional shortcuts that print the command they run; [the table in Your first migration](/sql-yodeler/getting-started/#the-same-steps-with-just) lists them. | | `yodel mcp` | read-only tools for a coding agent, over MCP ([Set up with a coding agent](/sql-yodeler/agents/#read-only-tools-yodel-mcp)). | | the JSON Schemas | for anything that parses `yodel`'s output: [JSON output](#json-output). | yodel runs [chant](https://intentius.io/chant/)'s CLI underneath (`chant init`, `chant emulator`, `chant run`, `chant approve`), so you never need to; [chant's CLI reference](https://intentius.io/chant/cli/overview/) is there for the curious. ## yodel --help ```console $ yodel --help Usage: yodel [options] Schema management and migrations for ClickHouse and Postgres. Commands: create make a new project from SQL Yodeler's template init adopt a live database, or record its baseline in another env emulator start, stop or check the local ClickHouse and Postgres (Docker) plan show what apply would change in an environment apply bring an environment to its declared schema or pending migrations approve approve the plan yodel apply would run (a person at a terminal) override override a policy rule for the current plan, with a reason new write the next migration from the declared schema checkpoint write a checkpoint: the newest recorded schema from nothing lint check the migrations against the lint rules test run tests/*.test.ts on databases built from the migrations status list applied, pending and out-of-order migrations report each environment's migrations side by side, and the audit log drift report declared objects changed out of band or gone docs write an HTML reference and a Mermaid ERD of the schema ci render the project's CI pipelines (GitHub, GitLab, Forgejo) config print each environment's settings, and prove a job cannot write rebase move a migration onto a new parent and recompute its schema repair record a new checksum for an applied migration, with a reason revert undo the newest applied migration from its recorded schemas cleanup drop the old tables and columns steps kept, once past their date mcp serve read-only tools to a coding agent (MCP, stdio) Options: -h, --help show this help -v, --version print the version yodel --help (or yodel help ) shows a command's options. Docs: https://intentius.io/sql-yodeler/ Exit codes (yodel --help lists a command's own): 0 done, or nothing to do 1 an error 2 something yodel will not do itself (a change that needs an Op it does not run, a manual step); for yodel drift, drift found 3 something outstanding: apply stopped at its approval gate, status found pending migrations, lint found errors 4 refused, nothing done: a checksum mismatch, an applied migration gone from the directory, an out-of-order migration, a fork or another problem in the migrations directory, a migration the environment it must run on first has not applied, an environment whose reader and writer are one identity, a command that writes run with an environment's reader 5 another apply holds the lock yodel plan exits 0 when migrations are pending: it reports, and a CI job that posts the plan on a pull request should not fail because the pull request adds a migration. To gate on pending migrations, use yodel status. ``` ## Exit codes `yodel --help` lists each command's options and exit codes. The exit codes mean the same across commands where they can: | Code | Meaning | |---|---| | 0 | done, or nothing to do | | 1 | an error | | 2 | something yodel will not do itself (a change that needs an Op it does not run, a manual step); for `yodel drift`, drift found | | 3 | something outstanding: apply stopped at its approval gate, status found pending migrations, lint found errors | | 4 | refused, nothing done: a checksum mismatch, an applied migration gone from the directory, an out-of-order migration, a fork or another problem in the migrations directory, a migration the environment it must run on first has not applied, an environment whose reader and writer are one identity, a command that writes run with an environment's reader | | 5 | another apply holds the lock | `yodel apply` exits with the code of its Op's Apply step, so a lock held by another apply is 5 there too. `yodel plan` exits 0 when migrations are pending: it reports, and a CI job that posts the plan on a pull request should not fail because the pull request adds a migration. To gate on pending migrations, use `yodel status`. ## JSON output With `--json`, a command prints one JSON document on stdout. Each document has a JSON Schema, shipped in the package's `schemas/` directory and published at `https://intentius.io/sql-yodeler/schemas/v1/.schema.json`: | Command | Schema | |---|---| | `yodel init --from --json`, `yodel init --baseline --json` | [`init`](https://intentius.io/sql-yodeler/schemas/v1/init.schema.json) | | `yodel plan --json` | [`plan`](https://intentius.io/sql-yodeler/schemas/v1/plan.schema.json): the versioned plan (`"mode": "versioned"`) or the declarative one | | `yodel apply --json` | [`apply`](https://intentius.io/sql-yodeler/schemas/v1/apply.schema.json) | | `yodel new --json` | [`new`](https://intentius.io/sql-yodeler/schemas/v1/new.schema.json) | | `yodel lint --json` | [`lint`](https://intentius.io/sql-yodeler/schemas/v1/lint.schema.json); `yodel lint --rules --json` prints [`lint-rules`](https://intentius.io/sql-yodeler/schemas/v1/lint-rules.schema.json) | | `yodel status --json` | [`status`](https://intentius.io/sql-yodeler/schemas/v1/status.schema.json) | | `yodel drift --json` | [`drift`](https://intentius.io/sql-yodeler/schemas/v1/drift.schema.json) | | `yodel rebase --json` | [`rebase`](https://intentius.io/sql-yodeler/schemas/v1/rebase.schema.json) | | `yodel repair --reason --json` | [`repair`](https://intentius.io/sql-yodeler/schemas/v1/repair.schema.json) | | `yodel cleanup --json` | [`cleanup`](https://intentius.io/sql-yodeler/schemas/v1/cleanup.schema.json) | Where a document asks for an approval, `approve` is the command a person runs, `yodel approve --plan `, and `chantApprove` is chant's own command for the same approval, for a script that records approvals itself. The version is in each schema's `$id` (`v1`). Within a version, fields are only added, so a reader ignores fields it does not know; removing a field or changing what one means is a new version. The exit code is the same with or without `--json`. `yodel mcp` serves the read-only commands to a coding agent, with these schemas: [Set up with a coding agent](/sql-yodeler/agents/#read-only-tools-yodel-mcp). --- # Glossary Source: https://intentius.io/sql-yodeler/glossary/ SQL Yodeler is built on chant. These are the chant terms the docs and yodel's output use, in yodel's terms. You need no other chant knowledge to use yodel. The terms are chant's, and [chant's documentation](https://intentius.io/chant/) covers each in full; the last column links the page for it. | Term | What it is in SQL Yodeler | In chant's docs | |---|---|---| | chant | the toolkit yodel is built on. yodel runs chant's CLI (the npm package `@intentius/chant`) underneath, so you do not run it yourself. | [chant](https://intentius.io/chant/) | | template | a starter project in this repository (`templates/clickhouse`, `templates/postgres`). `yodel create` makes a project from one, with chant's `init --from` underneath. See [Starting a project](/sql-yodeler/adoption/). | [From a template repository](https://intentius.io/chant/cli/init/#from-a-template-repository) | | the `sql` lexicon | chant's package for ClickHouse and Postgres (`@intentius/chant-lexicon-sql`). The declarations in `src/` use it, and yodel uses it to plan, diff and apply. | [The sql lexicon](https://intentius.io/chant/lexicons/sql/) | | `chant.config.ts` | the project's config: the dialect and one profile per environment. | | | profile | one environment in `chant.config.ts` (`sql.profiles.`): its server address, the variables its credentials are read from, and its databases or schemas. `yodel plan dev` uses the `dev` profile. See [Installing and configuring](/sql-yodeler/configuration/). | | | Op | a file in `ops/` that declares a run in steps. `ops/migrate-.op.ts` is what `yodel apply ` runs: plan, wait for approval, apply. See [Approval](/sql-yodeler/approval/). | [Ops](https://intentius.io/chant/guide/ops/) | | gate | the step in an Op that waits for a person's approval. `yodel apply` stops there (exit 3) until the plan is approved. | [Gate steps](https://intentius.io/chant/guide/ops/#gate-steps) | | plan digest | a hash of what `yodel apply` would do and the state it starts from (`jcs1-sha256:...`). An approval names a digest, and holds only while the plan still hashes to it. | [A gate approves a plan](https://intentius.io/chant/guide/ops/#a-gate-approves-a-plan-not-the-next-run) | | approval | a person's approval of a plan digest at a gate, recorded on `chant/lifecycle`. `yodel approve --plan `, the command yodel prints, shows the plan, asks for the environment's name and records it (chant's `approve` underneath; `--json` outputs carry chant's command as `chantApprove` for scripts). | [Gate steps](https://intentius.io/chant/guide/ops/#gate-steps) | | `chant/lifecycle` | the git branch where approvals are kept, as commits. Push it (`git push origin chant/lifecycle`) so CI sees an approval made on your machine. | [Gate steps](https://intentius.io/chant/guide/ops/#gate-steps) | | `chant run` | runs an Op. `yodel apply` runs the migrations Op itself, and the pipelines run the drift watch on its schedule; `yodel drift ` checks for drift by hand. | [chant run](https://intentius.io/chant/cli/run/) | | emulator | a local ClickHouse and Postgres in Docker for trying yodel: `npx yodel emulator up`. ClickHouse on `http://127.0.0.1:8123` (user `default`, no password), Postgres on `127.0.0.1:5432` (user `postgres`, password `chant`). | [chant emulator](https://intentius.io/chant/cli/emulator/) | | `ApplyOp` | the Op on the declarative path, where `yodel apply` applies the declared schema with no migrations. See [The two workflows](/sql-yodeler/workflows/). | [Applying](https://intentius.io/chant/lexicons/sql/applying/) | | `WatchOp` | the drift watch, an Op the pipelines run on a schedule. See [Drift](/sql-yodeler/drift/). | [Watching the lifecycle](https://intentius.io/chant/guide/watching-lifecycle/) | | wave | one environment of the apply pipeline, applied after the wave before it, behind its own gate policy. A tenant set is one wave over many databases or schemas. See [Waves](/sql-yodeler/approval/#waves-a-gate-per-environment). terragucci, also built on chant, uses the word for something else: a batch of Terraform roots applied behind one approval ([terragucci's definition](https://intentius.io/terragucci/concepts/glossary/#wave)). | [Op waves](https://intentius.io/chant/guide/op-waves/) | --- # Claims status Source: https://intentius.io/sql-yodeler/claims/ Each scenario claim (`scenarios/claims/`) is a promise the docs make, run against chant's emulator. A claim runs twice: plain, where it should pass, and broken (`BREAK=1`), where the claim's own check should catch the fault it plants; a claim of several parts is caught only when each part catches the fault planted in it. `npm run claims` runs them; see the comment at the top of `scenarios/run.ts`. The record was last written at 2026-10-10 21:02 UTC, at commit `9329873`. A row run in an earlier run keeps that run's commit and date. A row marked "not recorded" has not been run into the record. CI does not run the template claim, which needs a Forgejo with a runner. Each docs page names the claims that prove it, and `npm run check` fails when a page that is not a draft names one that does not pass plain and get caught broken here. The last column lists those pages. | Claim | Dialect | What it says | Plain | Broken | Commit | Date | Pages | |---|---|---|---|---|---|---|---| | adopt | ClickHouse | yodel init --from adopts a live database without touching it, and yodel plan then shows no change; on Postgres its policies, row-level security and grants too, on ClickHouse its dictionaries, functions, roles, users, row policies and grants; yodel init --baseline records the baseline, behind the gate, in a second environment that holds the same schema, and refuses one that differs | pass | caught | `9329873` | 2026-10-10 21:02 UTC | [access](/sql-yodeler/access/), [adoption](/sql-yodeler/adoption/), [from-other-tools](/sql-yodeler/from-other-tools/), [workflows](/sql-yodeler/workflows/) | | adopt | Postgres | yodel init --from adopts a live database without touching it, and yodel plan then shows no change; on Postgres its policies, row-level security and grants too, on ClickHouse its dictionaries, functions, roles, users, row policies and grants; yodel init --baseline records the baseline, behind the gate, in a second environment that holds the same schema, and refuses one that differs | pass | caught | `9329873` | 2026-10-10 21:02 UTC | [access](/sql-yodeler/access/), [adoption](/sql-yodeler/adoption/), [from-other-tools](/sql-yodeler/from-other-tools/), [workflows](/sql-yodeler/workflows/) | | new | ClickHouse | yodel new writes the next migration offline, with no dev database, and it applies; on Postgres, functions, procedures and triggers too, which lint checks | pass | caught | `c6f58a4` | 2026-10-10 20:29 UTC | [adoption](/sql-yodeler/adoption/), [getting-started](/sql-yodeler/getting-started/), [lint](/sql-yodeler/lint/), [migrations](/sql-yodeler/migrations/), [schema](/sql-yodeler/schema/), [workflows](/sql-yodeler/workflows/) | | new | Postgres | yodel new writes the next migration offline, with no dev database, and it applies; on Postgres, functions, procedures and triggers too, which lint checks | pass | caught | `c6f58a4` | 2026-10-10 20:29 UTC | [adoption](/sql-yodeler/adoption/), [getting-started](/sql-yodeler/getting-started/), [lint](/sql-yodeler/lint/), [migrations](/sql-yodeler/migrations/), [schema](/sql-yodeler/schema/), [workflows](/sql-yodeler/workflows/) | | lint | ClickHouse | yodel lint --replay replays the migrations into a fresh database and each gives the schema it recorded; offline, yodel lint fails a migrations directory with a fork (exit 3), naming both migrations; a checkpoint replays alone to its recorded schema, and a fresh environment starts from it; yodel revert undoes the newest migration behind the gate, with a hand-written step for its data step, back to its parent's recorded schema, and refuses a checkpoint or a migration before one; yodel test runs a project's tests on databases replayed from the migrations, a seed meeting the backfill after it, fails a case whose assertion does not hold, and refuses an environment with a history; on Postgres, a unique index over rows with duplicates is refused before the gate by its generated pre-check (exit 4, naming the statement and the count), and yodel lint flags it as data-dependent | pass | caught | `c6f58a4` | 2026-10-10 20:29 UTC | [getting-started](/sql-yodeler/getting-started/), [lint](/sql-yodeler/lint/), [migrations](/sql-yodeler/migrations/) | | lint | Postgres | yodel lint --replay replays the migrations into a fresh database and each gives the schema it recorded; offline, yodel lint fails a migrations directory with a fork (exit 3), naming both migrations; a checkpoint replays alone to its recorded schema, and a fresh environment starts from it; yodel revert undoes the newest migration behind the gate, with a hand-written step for its data step, back to its parent's recorded schema, and refuses a checkpoint or a migration before one; yodel test runs a project's tests on databases replayed from the migrations, a seed meeting the backfill after it, fails a case whose assertion does not hold, and refuses an environment with a history; on Postgres, a unique index over rows with duplicates is refused before the gate by its generated pre-check (exit 4, naming the statement and the count), and yodel lint flags it as data-dependent | pass | caught | `c6f58a4` | 2026-10-10 20:29 UTC | [getting-started](/sql-yodeler/getting-started/), [lint](/sql-yodeler/lint/), [migrations](/sql-yodeler/migrations/) | | pr-comment | ClickHouse | the pull request comment lists the pending migrations with each statement's class, and the digest the gate asks for | pass | caught | `c6f58a4` | 2026-10-10 20:29 UTC | [approval](/sql-yodeler/approval/) | | pr-comment | Postgres | the pull request comment lists the pending migrations with each statement's class, and the digest the gate asks for | pass | caught | `c6f58a4` | 2026-10-10 20:29 UTC | [approval](/sql-yodeler/approval/) | | approval | ClickHouse | a pending migration applies only after chant approve of its plan digest, and the history records that digest; a failing pre-migration check refuses it, and so does a policy rule, read at the base commit, unless an override is recorded for that digest, for a yodel revert as for an apply; the audit log derived from the history and the ledger accounts for every approval and apply, and names one removed; a reader and a writer that are one user are refused, and the reader, its password a minted token, cannot write; an approval is refused once the migration changed after it, and nothing is applied | pass | caught | `c6f58a4` | 2026-10-10 20:29 UTC | [approval](/sql-yodeler/approval/), [audit](/sql-yodeler/audit/), [configuration](/sql-yodeler/configuration/), [getting-started](/sql-yodeler/getting-started/), [migrations](/sql-yodeler/migrations/), [workflows](/sql-yodeler/workflows/) | | approval | Postgres | a pending migration applies only after chant approve of its plan digest, and the history records that digest; a failing pre-migration check refuses it, and so does a policy rule, read at the base commit, unless an override is recorded for that digest, for a yodel revert as for an apply; the audit log derived from the history and the ledger accounts for every approval and apply, and names one removed; a reader and a writer that are one user are refused, and the reader, its password a minted token, cannot write; an approval is refused once the migration changed after it, and nothing is applied | pass | caught | `c6f58a4` | 2026-10-10 20:29 UTC | [approval](/sql-yodeler/approval/), [audit](/sql-yodeler/audit/), [configuration](/sql-yodeler/configuration/), [getting-started](/sql-yodeler/getting-started/), [migrations](/sql-yodeler/migrations/), [workflows](/sql-yodeler/workflows/) | | resume | ClickHouse | an interrupted apply resumes where it stopped: a backfill step written as yodel's form (table, key, batch size, SQL) from its receipts, running each batch once, and a failed statement at that statement, never resending one that ran | pass | caught | `c6f58a4` | 2026-10-10 20:29 UTC | [from-other-tools](/sql-yodeler/from-other-tools/), [migrations](/sql-yodeler/migrations/), [steps](/sql-yodeler/steps/) | | resume | Postgres | an interrupted apply resumes where it stopped: a backfill step written as yodel's form (table, key, batch size, SQL) from its receipts, running each batch once, and a failed statement at that statement, never resending one that ran | pass | caught | `c6f58a4` | 2026-10-10 20:29 UTC | [from-other-tools](/sql-yodeler/from-other-tools/), [migrations](/sql-yodeler/migrations/), [steps](/sql-yodeler/steps/) | | out-of-order | ClickHouse | a migration merged late, before one already applied, is refused unless --allow-out-of-order, and never skipped | pass | caught | `c6f58a4` | 2026-10-10 20:29 UTC | [from-other-tools](/sql-yodeler/from-other-tools/), [migrations](/sql-yodeler/migrations/) | | out-of-order | Postgres | a migration merged late, before one already applied, is refused unless --allow-out-of-order, and never skipped | pass | caught | `c6f58a4` | 2026-10-10 20:29 UTC | [from-other-tools](/sql-yodeler/from-other-tools/), [migrations](/sql-yodeler/migrations/) | | rebuild | ClickHouse | ClickHouse: a sort-key change is a ClickHouseRebuildOp step inside the migration, never an ALTER or a drop; approved, it runs and keeps every row, and without an approval it does not run; yodel cleanup drops the old table it kept only after its retention date, behind an approval on a gate of its own, sealed under a sealed environment | pass | caught | `c6f58a4` | 2026-10-10 20:29 UTC | [steps](/sql-yodeler/steps/) | | drift | ClickHouse | yodel drift reports a declared object changed out of band, naming the property, and one dropped; on the versioned path it compares with the newest applied migration's recorded schema, so a pending migration is not drift | pass | caught | `c6f58a4` | 2026-10-10 20:29 UTC | [drift](/sql-yodeler/drift/), [getting-started](/sql-yodeler/getting-started/) | | drift | Postgres | yodel drift reports a declared object changed out of band, naming the property, and one dropped; on the versioned path it compares with the newest applied migration's recorded schema, so a pending migration is not drift | pass | caught | `c6f58a4` | 2026-10-10 20:29 UTC | [drift](/sql-yodeler/drift/), [getting-started](/sql-yodeler/getting-started/) | | declarative | ClickHouse | the declarative path: yodel plan shows the change against the live server and yodel apply makes it behind the plan-bound gate, for src/ declarations (on Postgres, functions, procedures and triggers, and access control: a role, a policy and grants, with a grant made by hand revoked; on ClickHouse, a dictionary, a function, and access control: a role, a user, a row policy and grants, with a grant made by hand revoked) and for an ORM's exported DDL | pass | caught | `c6f58a4` | 2026-10-10 20:29 UTC | [access](/sql-yodeler/access/), [configuration](/sql-yodeler/configuration/), [orm](/sql-yodeler/orm/), [schema](/sql-yodeler/schema/), [workflows](/sql-yodeler/workflows/) | | declarative | Postgres | the declarative path: yodel plan shows the change against the live server and yodel apply makes it behind the plan-bound gate, for src/ declarations (on Postgres, functions, procedures and triggers, and access control: a role, a policy and grants, with a grant made by hand revoked; on ClickHouse, a dictionary, a function, and access control: a role, a user, a row policy and grants, with a grant made by hand revoked) and for an ORM's exported DDL | pass | caught | `c6f58a4` | 2026-10-10 20:29 UTC | [access](/sql-yodeler/access/), [configuration](/sql-yodeler/configuration/), [orm](/sql-yodeler/orm/), [schema](/sql-yodeler/schema/), [workflows](/sql-yodeler/workflows/) | | topology | ClickHouse | ClickHouse: one migrations directory applies to a single node and to a Replicated database, and one to a sharded cluster with drift read there, each rendered for its topology | pass | caught | `c6f58a4` | 2026-10-10 20:29 UTC | [configuration](/sql-yodeler/configuration/), [topology](/sql-yodeler/topology/) | | template | ClickHouse | a project from the starter template, on Forgejo: apply only after approval, lint with replay and the plan comment on a pull request, and the approved change applied on merge; a sealed wave applies only on an approval sealed by a signer listed at the base, and a pr-review wave on the review of a writer other than the author; a pull request job cannot write, a forked migration fails lint and is annotated, a stale or hand-edited pipeline fails yodel ci --check, the CI image pinned by digest runs a pull request's jobs, a command token source mints the reader's password, and the drift watch keeps one tracking issue | pass | caught | `868ff97` | 2026-10-10 21:57 UTC | [forges](/sql-yodeler/forges/), [lint-and-plan](/sql-yodeler/lint-and-plan/), [notify](/sql-yodeler/notify/) | | template | Postgres | a project from the starter template, on Forgejo: apply only after approval, lint with replay and the plan comment on a pull request, and the approved change applied on merge; a sealed wave applies only on an approval sealed by a signer listed at the base, and a pr-review wave on the review of a writer other than the author; a pull request job cannot write, a forked migration fails lint and is annotated, a stale or hand-edited pipeline fails yodel ci --check, the CI image pinned by digest runs a pull request's jobs, a command token source mints the reader's password, and the drift watch keeps one tracking issue | pass | caught | `868ff97` | 2026-10-10 21:57 UTC | [forges](/sql-yodeler/forges/), [lint-and-plan](/sql-yodeler/lint-and-plan/), [notify](/sql-yodeler/notify/) | | template-github | ClickHouse | a project from the starter template, on GitHub Actions (act and a mock GitHub): apply only after approval, lint with replay and the plan comment on a pull request, and the approved change applied on merge; a sealed wave applies only on an approval sealed by a signer listed at the base, and a pr-review wave on the review of a writer other than the author; a pull request job cannot write, a forked migration fails lint and is annotated, a stale or hand-edited pipeline fails yodel ci --check, the CI image pinned by digest runs a pull request's jobs, a command token source mints the reader's password, and the drift watch keeps one tracking issue | pass | caught | `868ff97` | 2026-10-10 21:55 UTC | [forges](/sql-yodeler/forges/), [lint-and-plan](/sql-yodeler/lint-and-plan/), [notify](/sql-yodeler/notify/) | | template-github | Postgres | a project from the starter template, on GitHub Actions (act and a mock GitHub): apply only after approval, lint with replay and the plan comment on a pull request, and the approved change applied on merge; a sealed wave applies only on an approval sealed by a signer listed at the base, and a pr-review wave on the review of a writer other than the author; a pull request job cannot write, a forked migration fails lint and is annotated, a stale or hand-edited pipeline fails yodel ci --check, the CI image pinned by digest runs a pull request's jobs, a command token source mints the reader's password, and the drift watch keeps one tracking issue | pass | caught | `868ff97` | 2026-10-10 21:55 UTC | [forges](/sql-yodeler/forges/), [lint-and-plan](/sql-yodeler/lint-and-plan/), [notify](/sql-yodeler/notify/) | | template-gitlab | ClickHouse | a project from the starter template, on GitLab CI: apply only after approval, lint with replay and the plan comment on a merge request, and the approved change applied on merge; a merge request job cannot write, a forked migration fails lint and keeps its code quality report, a stale or hand-edited pipeline fails yodel ci --check, a project with ci.apply: false and no apply pipeline passes it and gets the plan comment, and the drift watch keeps one tracking issue | not recorded | not recorded | | | [forges](/sql-yodeler/forges/), [lint-and-plan](/sql-yodeler/lint-and-plan/), [notify](/sql-yodeler/notify/) | | template-gitlab | Postgres | a project from the starter template, on GitLab CI: apply only after approval, lint with replay and the plan comment on a merge request, and the approved change applied on merge; a merge request job cannot write, a forked migration fails lint and keeps its code quality report, a stale or hand-edited pipeline fails yodel ci --check, a project with ci.apply: false and no apply pipeline passes it and gets the plan comment, and the drift watch keeps one tracking issue | not recorded | not recorded | | | [forges](/sql-yodeler/forges/), [lint-and-plan](/sql-yodeler/lint-and-plan/), [notify](/sql-yodeler/notify/) | | lock-retry | ClickHouse | Postgres: a statement blocked behind another session's lock times out at lock_timeout and is retried until it succeeds; Postgres and ClickHouse with Keeper: two applies at once never send a statement twice, the second waits naming the holder, and --stand-down steps aside for a newer commit | pass | caught | `c6f58a4` | 2026-10-10 20:29 UTC | [migrations](/sql-yodeler/migrations/) | | lock-retry | Postgres | Postgres: a statement blocked behind another session's lock times out at lock_timeout and is retried until it succeeds; Postgres and ClickHouse with Keeper: two applies at once never send a statement twice, the second waits naming the holder, and --stand-down steps aside for a newer commit | pass | caught | `c6f58a4` | 2026-10-10 20:29 UTC | [migrations](/sql-yodeler/migrations/) | | column-change | Postgres | Postgres: a column rename runs as a PostgresMigrationOp step (expand, backfill, switch, contract) and keeps every value; a step that fails part way keeps its work and resumes from its receipts | pass | caught | `c6f58a4` | 2026-10-10 20:29 UTC | [steps](/sql-yodeler/steps/) | | waves | ClickHouse | the apply pipeline runs one wave per environment, in order, each behind its gate policy read from the base commit, applies only what the wave before applied, and applies a tenant set's migrations to every tenant behind one gate; a sealed wave counts only an approval sealed by a signer the base commit lists | pass | caught | `c6f58a4` | 2026-10-10 20:29 UTC | [approval](/sql-yodeler/approval/), [workflows](/sql-yodeler/workflows/) | | waves | Postgres | the apply pipeline runs one wave per environment, in order, each behind its gate policy read from the base commit, applies only what the wave before applied, and applies a tenant set's migrations to every tenant behind one gate; a sealed wave counts only an approval sealed by a signer the base commit lists | pass | caught | `c6f58a4` | 2026-10-10 20:29 UTC | [approval](/sql-yodeler/approval/), [workflows](/sql-yodeler/workflows/) |