What you pay, and when#

choudoufu puts a resource’s identity on the resource, as two AWS tags, so that a state file stops being the record of what you own. That swap has a price. This page is the measured statement of it, for someone deciding whether to put an estate on this fork.

The price is not one number, because it is not paid on every run. It depends on which of three things the run is doing:

Every figure on this page describes choudoufu v0.5.0. Each one names its fixture, its commit, and whether it came from the pinned AWS emulator or from a real AWS account. The two are not interchangeable and are never combined.

There is currently no wall-clock figure on this page. The three real-AWS sessions that produced one were comparing a cached plan against an uncached one, and Wall clock sets out why they are withdrawn rather than restated.

With no live block, nothing at all#

A configuration with no live block and no estate.chdf.hcl sidecar runs as stock OpenTofu. That is measured, not promised.

Same generated estate, same emulator, same session, every column an empty plan, three runs each, no variance in any column:

The 79-instance column was re-run at b20a144ab0 for this page; the 301-instance column is the ruling’s and has not been re-run since. Both oracles are behind the current pin, which live/oracle-versions.json puts at terraform 1.16.0 and tofu 1.12.6. Nothing in this table has been re-run against those.

OpenTofu is in that table because choudoufu is an OpenTofu fork and Terraform is not OpenTofu. Without the middle row, any difference between the top and bottom rows could not be attributed to this fork rather than to the Terraform/OpenTofu split.

The code half holds too. Every fork addition to plan and apply sits behind a guard that a missing live block turns off. Seven things do run unconditionally; they are enumerated in the ruling, and the one among them that can change a verdict changes it in the accepting direction, so a configuration stock refuses can succeed here and never the reverse. Method, per-guard reading and raw values: [the stateful-equivalence measurement (#588)](https://github.com/INTENTIUS/choudoufu/blob/main/the stateful-equivalence measurement (#588)).

Planning an adopted estate#

Turn the live block on, migrate the estate, and plan it the way an operator plans on an ordinary Tuesday. Same fixture, same emulator, three runs each, every plan exit 0 with No changes:

157 against 150 is +4.7%, and the residual is seven calls rather than a percentage, because the two sides can be diffed action by action. Of stock’s 150 calls, 148 across 18 AWS actions are matched exactly - same actions, same counts:

ListAttachedRolePolicies                32    DescribeVpcAttribute        3
GetRole                                 21    DescribeSecurityGroups      2
GetRolePolicy                           20    DescribeVpcs                1
GET /{bucket}/...                       12    DescribeSubnets             1
ListRolePolicies                        11    DescribeRouteTables         1
GetPolicy                               10    DescribeNetworkAcls         1
GetPolicyVersion                        10    ECS DescribeClusters        1
GetInstanceProfile                      10    ECS DescribeServices        1
GET ?maxitems&name&type  (Route 53)     10    ECS DescribeTaskDefinition  1

That is the refresh, and it is the AWS provider’s own Read implementations. Stock invokes them on the same resources when it refreshes; nothing in this fork adds to them or can subtract from them. Stock’s other two calls are GetCallerIdentity and GetUser, resolving the account.

choudoufu’s other nine are those same two, doubled, plus seven that are only on this side:

The first three are the estate-scoped sweep, and they do not shrink further without giving up removal coverage for those types. The last four are this fork’s own structure and have nothing to do with the sweep.

The 157 has now been produced three separate times on this fixture and pin: by #627, which landed the narrowing that produced it; by [the slicing measurement (#584, corrected by #634)](https://github.com/INTENTIUS/choudoufu/blob/main/the slicing measurement (#584, corrected by #634))’s re-measure at 5ff7f43f5b, which reproduced its seven-call residual call for call; and by the run reported here, at b20a144ab0. Reproduce it with TF_FLOCI_TEST=1 go test ./internal/live/statefulcost/, which is also where the no-live-block table above comes from.

Until 09d180f921 a plan enumerated the whole admission table on every run, about 512 native-leg list calls whatever the estate contained, and this same fixture read 710 where it now reads 157. The 301-instance emulator figure was taken before that change and has not been re-run, so it describes a plan that no longer exists.

The same comparison on real AWS, at 79 and 745 resources#

Real AWS in account ...3429 and region us-east-2 on a main containing all of the concurrency and narrowing work. Both sides no-change plans on every run:

Re-measured at d359210978. At 745 resources choudoufu makes fewer provider requests than stock, in both sessions, which supersedes the +7 this table carried before. Per-operation the two are near-identical: ListAttachedRolePolicies 325 against 324, GetRole 215 against 211. The one real difference is DescribeTaskDefinition, 21 against 10.

Note before anything else that the 79-resource row disagrees with the emulator row above it: +16 against +7 on the same fixture at the same scale. Do not average them or pick the flattering one. They come from different instruments - the emulator figure counts every HTTP request through a proxy, this one counts only what the AWS provider itself logs - from different accounts, and from a comparison where the two sides plan different directories. Both are reported; neither is a correction of the other.

The residual shrinks in absolute terms between the two scales, which is not what a fixed overhead does, and the reason is that the two sides are not merely one adding to the other. At 745 resources choudoufu makes fewer IAM calls than stock - ListAttachedRolePolicies 320 against 324, GetRole 210 against 215, GetRolePolicy 200 against 204, ListRolePolicies 110 against 114 - and more ECS calls, DescribeTaskDefinition 31 against 10 plus a ListTaskDefinitions, a ListServices, two ListRoles and one extra GetCallerIdentity. Route 53 is identical on both sides at both scales: GetHostedZone 101, ListResourceRecordSets 100, ListTagsForResource 1. Why the IAM side comes out lower is not explained by that run, and this page is not going to invent a reason for it.

Two conditions travel with that table and change how it should be read.

It counts provider-mediated requests only. The figures come from terraform-provider-aws’s own HTTP Request Sent log entries. choudoufu’s Cloud Control and Tagging clients log no line per request, so their HTTP calls are not in the 1413. What is known about them is a type count rather than a call count: 0 types went via Cloud Control, and 31 went through the estate-filtered tagging sweep, which is one GetResources for all 31 plus pagination. That share is small and unmeasured, and the run says so itself. The emulator tables higher up the page count every request through a proxy, so the two instruments have different denominators and their numbers should not be subtracted from one another.

The two sides are not planning the same directory. Stock plans its own converged state after the cold deploy and before anything migrates it; choudoufu plans the migrated estate. That is the honest like-for-like available on real AWS, and it is a weaker control than the emulator’s, where both sides plan the same estate through the same proxy.

If you want a number for your estate, measure your estate. The composition above is a property of what this fixture declares rather than of this fork.

The record store, which no call count used to see#

Every managed instance also has a record: the arguments the provider never echoes back, sensitivity marks, taint, the deposed key. It lives in a local directory, an SSM parameter or an S3 object, and it therefore never crosses the AWS endpoint the counting proxy stands in front of. So until #636, every “a plan costs N calls” figure in this repository was a figure about one of the two things a plan does, presented as if it were both.

Counted, at 79 instances with 78 records: 377 round trips, over 80 distinct keys, 297 of them re-reading a key an earlier trip in the same plan had already read. Each accessor on the record store decoded the same physical key and each went to the store for itself. Stock OpenTofu pays one read for the same information.

It now pays one too. A bulk read plus a run cache that switches itself off permanently the first time anything writes through it takes the same plan to 1 trip, one GetAll, with the three plan outputs byte-identical to the 377-trip baseline’s, run for run, reproduced here at b20a144ab0:

run 1: aws-calls 157, record-trips 1
run 2: aws-calls 157, record-trips 1
run 3: aws-calls 157, record-trips 1

On SSM an estate of N records costs ceil(N/10) API calls rather than N, because the bulk path keeps the values the paged call already returned. S3 cannot bulk-fetch bodies at all, so its floor stays 1 + N and only the seam moved. See Storage for which store to pick.

Migration writes no markers by hand#

The cost that would decide an adoption is human, not API: how much of an existing estate needs a tofu-address typed by a person before the first plan.

Measured on real AWS rather than an emulator, at 745 resources in us-east-2, recorded in live/gauntlet.json’s live_cert block at 1d06e1d177:

migrate is the write-side cost and the one stage that is genuinely serial: 335 unbatched tag writes, one per resource, plus 190 throttle responses all absorbed by the SDK’s own backoff. It is paid once.

Nobody wrote a marker by hand. The 410 skipped are untaggable by the provider’s own schema and need no marker at all: their identity composes from an already-stamped parent, which is a role name plus an inline-policy name, or a zone ID plus a record name and type.

This matters more than the same ratio measured earlier on the emulator, because that earlier run could not test its own predicted bottleneck. The documented hard limit is that a count/for_each instance is never offered for adoption by content matching, and the generator that #566 measured emitted no count and no for_each anywhere. It reported that scoping limit itself. #574 added root-level count, root-level for_each over a map, and a module call whose body also carries count, and the run above is that estate. live-import reads each instance’s identity straight out of the state file, so it never reaches the content-matching wall that limit describes.

What that leaves is the day the state file is gone or wrong, which is the adoption path proper and the one place the expensive sweep is worth its price. Migrate an existing estate covers both routes.

The old state file stops being a safe fallback#

This is the part of migration that costs something, and it is worth knowing before you migrate rather than after.

live-import stamps markers onto live objects. The stock state file it read was taken before that stamp, so it does not know the tags exist. Run stock terraform plan in that same untouched directory afterwards and it refreshes, finds two tags the configuration does not declare, and proposes removing them.

Measured here at b20a144ab0, on the same 79-instance estate, with a control run immediately before the stamp:

ROUND TRIP before live-import:            terraform plan is empty
ROUND TRIP after live-import -approve:    Plan: 0 to add, 38 to change, 0 to destroy

All 38 changes are the same shape, and this is the whole of one of them:

~ tags = {
    - "tofu-address" = "aws_ecs_cluster.main" -> null
    - "tofu-estate"  = "sfcost-scd" -> null
  }

Applying that un-migrates the estate, and it reads as routine tag drift while it does so. 38 of 79 instances at this scale, 137 of 301 at the next one.

choudoufu refuses this run; stock cannot. Since #613 a state-backed plan under this fork that would strip a migrated estate’s markers is computed and rendered in full, so the operator sees exactly the drift stock would have shown, and then refused rather than applied. The refusal covers apply and plan -out followed by apply on the saved file, because the damaging form is an unattended apply -auto-approve where a warning would be correct and unread. CHOUDOUFU_UNMIGRATE=<estate> downgrades it to a warning when backing out is what you actually want, and it takes an estate name rather than an on/off value so that a setting left behind in CI cannot cover an estate migrated a year later.

None of that helps a directory running stock. Delete or archive the old state file as part of the migration, and treat any surviving stock directory pointing at the same estate as the hazard it now is.

Slicing: the 30.6x figure is withdrawn#

The question this page was written to answer was whether it is still worth slicing a terralith into separate states under choudoufu, and the published answer was that slicing was catastrophic here: stock cost 148 calls whether it was one state or eight, and choudoufu cost 4530 at eight, a 30.6x ratio.

Do not quote that number. It was measured on a configuration where choudoufu plan refused. The bench wrote its own provider block setting skip_requesting_account_id, which breaks ECS identity resolution, and every choudoufu plan in that matrix exited 1 while its call count was written up as a clean plan’s cost.

Re-measured at 5ff7f43f5b against the same pin, scale 1, every plan exiting 0 with No changes:

Two things in the old answer were wrong, and the second is the one worth carrying away.

Stock is not cost-neutral under slicing. Its “148 whether one state or eight” was itself an artifact of the same provider block. Stock is 148 + 2k, two calls per slice to resolve the account. choudoufu pays about six per slice, so an extra state costs roughly four calls more here than it costs stock.

The broken configuration was not uniformly more expensive, and past k≈5 it was cheaper. An A/B at one commit and pin, varying only the provider block, puts the overhead at +14, +10 and −14 calls at k=1, 2 and 8. Two constants of opposite sign make it: the refusal costs +18 in whichever single slice holds the ECS layer, however large k is, while failing to resolve the account saves 4 in every slice. Net 18 − 4k, zero near k=4.5. So the error depended on k and changed sign, which is why the old ratio could not be scaled or salvaged.

But the sweep did not go away, and it is still per state#

The flat leg that made the old answer alarming is still there. The estate-wide native sweep still does not scale down with a slice’s type count: 512 calls per slice at every k measured, which is 4096 summed across eight states. A slice declaring five types pays what the whole estate pays, and fractionally more, because the sweep builds its universe by subtracting the types a configuration declares from the admission table.

What changed is who pays it. 09d180f921 took that leg off the steady-state plan path when the estate has its own evidence to narrow by, and left it everywhere else. A plan with no record store, a store that will not list, or an empty store still takes the full universe, because every gate fails toward doing the work.

So the honest split, and it is a split rather than a verdict:

• As a claim about day-2 planning, “don’t slice under choudoufu” is dead. A steady-state plan of a sliced estate costs 1.21x stock’s at eight states.
• As a claim about adoption and recovery, it stands. Every run that actually sweeps pays about 512 calls per state, and eight states means eight sweeps.

What this page does not claim#

There is no crossover. The hypothesis this work was filed under was that choudoufu’s plan is a high fixed cost plus a low marginal cost, so that past some estate size the two curves cross and choudoufu wins outright. The first half is right and the second is not. Fitted to the two scales measured on a full-sweep plan:

stock                = 1.84N + 5      (150 at N=79, 558 at N=301)
choudoufu, migrated  = 1.99N + 553    (710 at N=79, 1152 at N=301)

The fixed cost is real. The marginal cost is about 8% higher rather than lower: a migrated instance is read by the projection and also pays its share of the sweep. The ratio therefore falls with N - 4.7x at 79 instances, 2.1x at 301 - while the absolute difference never closes. There is nothing to cross.

Two caveats on those two lines, in opposite directions. Two points determine a line exactly, so that fit has no residual and is a description of two measurements rather than a tested model. And since 09d180f921 it no longer describes a steady-state plan at all: it describes a run that sweeps the whole admission table, which today means an adoption or a recovery.

Nothing in the seconds crosses either. As measured at 5dc10cc781 the wall-clock ratio narrowed the same way the call ratio does, roughly 5x at 79 resources to roughly 4x at 745, and both sides climbed. A narrowing ratio is not an approaching crossover, and this page will not draw one until a measurement brackets it on both sides. Both of those ratios are superseded by the fix described below and have not been re-measured on real AWS.

No claim is made about another provider. Every figure here is AWS.

No claim is made from one resource type’s slope. live/plan-budget.json ratchets an aws_s3_bucket estate at 22 calls per instance; the terralith above reads 1.84. Same code, twelve times apart, because aws_s3_bucket has an unusually chatty Read. Extrapolating from somebody else’s resource type will be wrong by whatever the ratio between the two providers’ Read implementations happens to be.

Wall clock: withdrawn, because the comparison was not like for like#

Every wall-clock ratio this page published at 745 resources is withdrawn. Three sessions had measured 2.95x, 3.53x and 2.05x on the median, and the page had begun offering a mechanism for them. The comparison underneath was invalid.

The harness ran stock in one directory and choudoufu in another. Stock’s directory holds terraform.tfstate. choudoufu’s holds .tofu-records and no state file at all: live-import reads stock’s state file to perform the migration and never writes one into the estate it migrates to.

So the measurement was not stock against this fork. It was a cached plan against an uncached one. choudoufu read all 745 objects live because nothing had told it any of them were already known, which is what a plan does when its cache is missing rather than stale.

This is a defect in the test harness, not a discovery about the fork, and the figures cannot be salvaged by reinterpreting them. They are withdrawn rather than restated.

What this fork actually keeps, and what the test was missing#

choudoufu is stock OpenTofu plus identity hooks. It keeps the ordinary state file, as a cache, and adds three pieces that live in the cloud: identity as two tags on the resource, values in a record store, and effects as receipts. The state file is demoted from being the record of what you own to being a copy you are allowed to lose or to find stale. Demoted is not deleted, and a cache you never write is not a cache.

Two things were therefore missing from the comparison. choudoufu’s estate had no state file to cache into, and its record store was configured as record_store "local", a directory on disk, so the values piece was never exercised against the cloud either. Of the three pieces, only identity - the tags - was genuinely under test.

What survives#

The API call counts survive: they were counted per request on both sides and do not depend on the cache being present. choudoufu issues fewer provider requests than stock at 745 resources in all three sessions.

The head-of-line defect survives (#683). A block profile puts 46.14 s of a 51 s plan in one channel receive, and one throttled read stalls the whole read pass regardless of why the reads are being made. That is a real defect in the read pass’s slot accounting. What changes is its importance: it was stalling a pass that should not have been reading 745 objects in the first place.

A corrected comparison, with choudoufu given the state file it is designed to keep and a record store in the cloud, has not been run yet. Nothing replaces these figures until it has.

And an emulator cannot answer this question#

A wall clock measured against the pinned emulator grades the machine the test ran on rather than this repository’s code, which is why live/plan-budget.json records one and never gates on it. Every second on this page is real AWS for that reason. live/FLOCI.md sets out the rule and the three other questions an emulator-backed measurement cannot answer.

What evaluating this costs in money#

Effectively $0.00, against a $15 ceiling, for the whole 745-resource run. Every write the fixture makes is free - IAM, VPC, subnet, security group, Route 53 record changes, ECS cluster, service and task definitions - and desired_count = 0, so no Fargate task ever ran. Both hosted zones were deleted well inside the twelve hours below which AWS does not charge for one.

Teardown was confirmed by listing rather than inferred from an exit code, and then verified again independently through the AWS CLI: the account is back to its baseline, and the 21 ARNs still answering the run’s own tag were described one at a time to confirm each was INACTIVE at zero running and zero desired.

Where the mechanism is#

This page is the decision. What a plan costs is the mechanism: the two terms a plan is made of and how each one grows, the per-leg split at three scales, and the two concurrency bounds you can turn down when a real account starts answering Rate exceeded.