Skip to content
chant

Apply Conformance Suite

describeApplyConformance, from @intentius/chant-test-utils, is the suite every applier runs against its own transport mocks. It checks the tri-state and reason vocabulary documented in Implementing Apply against your implementation.

Run the conformance suite

Section titled “Run the conformance suite”

From your lexicon’s own test file, where your transport mocks live:

import { describeApplyConformance } from "@intentius/chant-test-utils";


describeApplyConformance({
  lexicon: "gcp",
  scenarios: [{
    name: "a manifest with one mapped and one unmapped kind",
    plan: [{ kind: "StorageBucket", name: "mapped" }, { kind: "SQLInstance", name: "unmapped" }],
    run: () => gcpApply(args, undefined, mockHttp).then(toApplyResult),
    expectApplied: ["StorageBucket/mapped"],
    expectNotAttempted: ["SQLInstance/unmapped"],
  }],
});

The config takes three scenario lists, and which assertions run depends on which you supply.

FieldTypeDrives
lexiconstringTest titles.
scenariosApplyScenario[]Assertions 1, 2, 3.
pruneScenariosPruneScenario[]Assertion 4. Optional, and an applier with no prune has nothing to assert.
idempotenceScenariosIdempotenceScenario[]Assertion 5. Optional, but every applier claims idempotence, so its absence is a gap.

An ApplyScenario is { name, plan, run, expectApplied?, expectPruned?, expectNotAttempted? }. plan is the ApplyRef[] the applier was handed; the three expect* lists name resources as kind/name. run returns either the apply: "v1" envelope or a partial { applied, pruned, notAttempted }, so an un-migrated applier can still be held to assertions 1 through 3.

A PruneScenario is { name, run, ownedOrphan, foreign }. Its run returns { result, deletes }, where deletes is every delete target the transport was asked for. ownedOrphan and foreign are substrings matched against those targets: exactly one delete must match the first, and none may match the second.

An IdempotenceScenario is { name, run }, and run returns { first, second } from applying the same plan twice against a transport that remembers the first.

What it proves:

  1. Shape — the buckets are disjoint. A resource cannot be both written and skipped.
  2. Total reasons — every not-attempted entry names a legal reason, every applied entry a legal action.
  3. Nothing is dropped — every resource in the plan is accounted for. This is the suite’s reason to exist.
  4. Owned-only prune — given a foreign resource and an owned orphan in the same scope, exactly one delete is issued, against the orphan. Asserted on the transport, not the return value.
  5. Idempotence — the same plan applied twice reports no created the second time.

Assertion 4 is on the transport deliberately. An applier that returns a tidy result while issuing a delete against a stranger’s resource passes every other check, which is precisely what the ARM target did when owned-only ran az deployment --mode Complete.

Six appliers run the suite today. lexicons/gcp/src/op/activities/gcp-apply.test.ts, lexicons/grafana/src/op/activities/grafana-apply.test.ts, lexicons/sql/src/op/activities/clickhouse-apply.test.ts and lexicons/sql/src/op/activities/postgres-apply.test.ts use all three scenario lists; lexicons/azure/src/op/activities/az-apply.test.ts and lexicons/fly/src/op/activities/fly-apply.test.ts are the others.

See also

Section titled “See also”