Composites

The otel lexicon writes collector config. Running a collector is the platform’s job, so platform composites live in the platform lexicons and use collectorYaml() to render a declared config into whatever file the platform mounts.

NodeAgent

NodeAgent declares the collector config a per-node agent runs on Kubernetes, one collector per node as a DaemonSet. It takes OTLP from the node’s pods, reads the node’s own telemetry, puts Kubernetes metadata on all of it and hands it on, usually to a gateway.

import { NodeAgent, OtlpExporter, PrometheusExporter } from "@intentius/chant-lexicon-otel";

const gateway = new OtlpExporter({ name: "gateway", endpoint: "otel-gateway.observability.svc:4317", tls: { insecure: true } });
const scrape = new PrometheusExporter({ endpoint: "0.0.0.0:8889" });

export const agent = NodeAgent({ exporters: [gateway], metricExporters: [scrape], clusterName: "prod-eu-1" });

Prop	Default	What it does
exporters	required	Where traces and logs go, and metrics unless metricExporters is set
metricExporters	exporters	Where metrics go, e.g. a prometheus exporter scraped on each node
clusterName	none	Adds a resource processor that sets k8s.cluster.name
nodeNameEnv	K8S_NODE_NAME	The environment variable holding the node name; k8sattributes filters to this node’s pods by it
hostMetrics	on, every 30s	hostmetrics with the CPU, memory, load, filesystem and network scrapers, reading the host root at /hostfs ({ interval, rootPath })
containerLogs	on	filelog over /var/log/pods with the container parser, leaving out the agent’s own container ({ selfContainer }, default otel-collector)
kubeletStats	off	kubeletstats against this node’s kubelet with the service account ({ interval })
memoryLimitMib	400	memory_limiter’s limit; the spike limit is a quarter of it
healthCheck	on	A health_check extension on 0.0.0.0:13133

Every pipeline runs memory_limiter, k8sattributes, resourcedetection (env, system), the cluster-name resource processor when there is one, and batch, in that order. k8sattributes matches a record to its pod by k8s.pod.ip, then k8s.pod.uid (which the container log parser sets), then the connection. There is no tail sampling: an agent sees only its own node’s spans of a trace, so sample on the gateway. The members are named (agent.otlp, agent.traces, …), and Object.values(agent.members) is the entity list collectorYaml and the platform composites take.

The pod has to provide the node name in nodeNameEnv (from spec.nodeName), the host root mounted read-only at /hostfs, /var/log/pods mounted read-only, and a service account that may read pods, namespaces, nodes and replicasets (and nodes/stats with kubeletStats). The k8s lexicon’s OtelCollector takes the members as its config and provides all of it, worked out from the config: the variable from spec.nodeName, both host mounts read-only, group 0 so its non-root user can read the root-owned log files, and the RBAC and ports (see node access). Its WK8605 check reports a hand-written workload that leaves one out.

chant init --lexicon otel --template k8s-agent scaffolds a project around it.

RedMetrics

RedMetrics declares rate, errors and duration (RED) metrics from traces, served for Prometheus to scrape. Spans come in on an otlp receiver, go through memory_limiter and batch to a spanmetrics connector and a servicegraph connector, and both feed a metrics pipeline that ends at a prometheus exporter on 0.0.0.0:8889. Traces can go on to a backend as well.

import { RedMetrics, OtlpExporter } from "@intentius/chant-lexicon-otel";

const tempo = new OtlpExporter({ name: "tempo", endpoint: "tempo:4317", tls: { insecure: true } });
export const red = RedMetrics({ traceExporters: [tempo], spanMetrics: { namespace: "shop", histogram: { unit: "s" } } });

Prop	Default	What it does
receivers	an otlp receiver on 4317 and 4318	Where spans come from; a connector from another traces pipeline works too
traceExporters	none	Where traces go besides the connectors
exporter	prometheus/red on 0.0.0.0:8889	The prometheus exporter the metrics are served on
spanMetrics	the connector’s defaults	The spanmetrics config: namespace, dimensions, histogram unit and buckets
serviceGraph	on	The servicegraph connector, or its config; false leaves it out
memoryLimitMib	80% of the container’s memory	memory_limiter’s hard limit; the spike limit is a quarter of it
name	red	The instance name of the connectors, the default exporter and the pipelines (traces/red, metrics/red)

redMetricsNames(red) returns the Prometheus names it serves, read with spanMetricsNames() and serviceGraphNames(). The grafana lexicon’s RedDashboard and the prometheus lexicon’s RedAlerts take the connector and the exporter, so the dashboard and the alerts follow a renamed namespace:

import { RedDashboard } from "@intentius/chant-lexicon-grafana";
import { RedAlerts } from "@intentius/chant-lexicon-prometheus";

export const dashboard = RedDashboard({ spanMetrics: red.spanMetrics, exporter: red.exporter, datasource: prometheus });
export const alerts = RedAlerts({ spanMetrics: red.spanMetrics, exporter: red.exporter });

Both build their PromQL with spanMetricsRedQueries(names, { range }) from this lexicon’s metric-names module, so a panel and the alert a responder opens it from run the same expression. The dashboard reads it over $__rate_interval, the alerts over their rateWindow.

GenAiPipeline

GenAiPipeline(options) declares the same config genAiPipeline(options) returns, with each entity under a member name: otlp, traces, sampled (with sampling), genAiTraces, genAiMetrics, sdkMetrics (with clientMetrics), logs and health. The options and their defaults are the preset’s; see GenAI pipeline.

import { GenAiPipeline, PrometheusExporter } from "@intentius/chant-lexicon-otel";

const scrape = new PrometheusExporter({ endpoint: "0.0.0.0:8889" });
export const genai = GenAiPipeline({ metricExporters: [scrape], clientMetrics: "derive" });

The prometheus lexicon’s GenAiRules reads the metric names from genAiMetrics(options) called with the same options.

OtlpCollector

OtlpCollector(options) declares the config otlpCollector(options) returns (see the helpers below) with each part as a member: otlp, memoryLimiter, batch, debug when no exporters are given, health unless healthCheck is false, and traces, metrics and logs for the signals asked for.

import { OtlpCollector, OtlpExporter } from "@intentius/chant-lexicon-otel";

const backend = new OtlpExporter({ name: "backend", endpoint: "collector.example.com:4317" });
export const collector = OtlpCollector({ exporters: [backend], signals: ["traces", "logs"] });

TailSamplingTier

TailSamplingTier declares the collector config of a tail sampling tier for hosts other than Kubernetes (VMs, Docker, Fly, ECS). On Kubernetes, the k8s lexicon’s OtelCollectorGateway and its WK8601 to WK8603 checks cover the same ground. Tail sampling needs every span of a trace in one collector, so the agents in front send through a loadbalancing exporter that routes by trace id; tailSamplingLoadBalancer(resolver, name) declares it.

import { TailSamplingTier, tailSamplingLoadBalancer, OtlpExporter } from "@intentius/chant-lexicon-otel";

const tempo = new OtlpExporter({ name: "tempo", endpoint: "tempo:4317", tls: { insecure: true } });
export const sampler = TailSamplingTier({ exporters: [tempo], slowerThanMs: 500, percentage: 5 });

// On each agent, in front of the tier:
export const toSampler = tailSamplingLoadBalancer({ dns: { hostname: "sampler.internal" } }, "sampler");

Prop	Default	What it does
exporters	one debug exporter	Where the kept traces go
receivers	an otlp receiver on 4317 and 4318	Where spans come from
decisionWait	10s	How long after a trace’s first span the decision is made
errors	on	Keep every trace with a span in error (a status_code policy)
slowerThanMs	1000	Keep every trace at least this long (a latency policy); false for none
percentage	10	Keep this share of the rest (a probabilistic policy); false for none
policies	none	More tail_sampling policies, after the three above
numTraces	the collector’s 50000	How many traces to hold in memory
memoryLimitMib	80% of the container’s memory	memory_limiter’s hard limit
healthCheck	on	A health_check extension on 0.0.0.0:13133

The pipeline is otlp, memory_limiter, tail_sampling, batch, then the exporters. Compute span metrics before this tier, on the agents or with RedMetrics, or they count only the traces it keeps. tailSamplingLoadBalancer leaves the exporter’s TLS on; pass { otlp: { tls: { insecure: true } } } as its third argument on a private network.

collectorYaml(entities)

Returns exactly the YAML the otel serializer would emit for the given entities, including the # chant: pin lines for custom components. Pass the components and pipelines; a component a pipeline references is included automatically.

import { OtlpReceiver, DebugExporter, Pipeline, collectorYaml } from "@intentius/chant-lexicon-otel";

const otlp = new OtlpReceiver({ protocols: { grpc: { endpoint: "0.0.0.0:4317" } } });
const debug = new DebugExporter({});
const yaml = collectorYaml([new Pipeline({ signal: "traces", receivers: [otlp], exporters: [debug] })]);

buildCollectorConfig(entities) returns the same config as a plain object, before it is printed.

Helpers for platform composites

The platform composites share a few helpers from this lexicon, and a composite for another platform can use them the same way.

otlpCollector({ exporters, signals, healthCheck }) returns the entities of a small OTLP collector: an otlp receiver on 4317 (gRPC) and 4318 (HTTP), memory_limiter then batch, the given exporters (one debug exporter by default), a health_check extension on 13133 unless healthCheck is false, and one pipeline per signal (traces, metrics and logs by default). Pass the result to collectorYaml.

genAiPipeline(options) returns the entities of a collector for GenAI workloads in the same form: content removal on traces and logs, and span and token metrics from every GenAI span. See GenAI pipeline.

collectorEndpoints(config) reads a built config and returns the ports its receivers listen on, each with a name usable as a Kubernetes port name (otlp-grpc, otlp-http), and the health_check port and path when the service enables one on an address other than localhost. The composites publish ports and set up health checks from this, so both follow whatever config the caller declares.

COLLECTOR_IMAGE is the contrib collector image at the version the built-in components are typed against, and COLLECTOR_CONFIG_PATH is /etc/otel/config.yaml, where every platform composite puts the rendered config.

import { buildCollectorConfig, collectorEndpoints, collectorYaml, otlpCollector } from "@intentius/chant-lexicon-otel";

const entities = otlpCollector({ signals: ["traces"] });
const yaml = collectorYaml(entities);
const { ports, healthCheck } = collectorEndpoints(buildCollectorConfig(entities).config);
// ports: otlp-grpc 4317, otlp-http 4318; healthCheck: port 13133, path "/"

Platform composites

Each composite takes either exporters and signals for the default config, or config with the full set of entities.

Composite	Lexicon	What it runs
DockerOtelCollector	docker	A Compose service, with the config inline in a top-level Compose config
OtelCollector	k8s	A DaemonSet agent on any cluster, with a node-local Service
OtelCollectorGateway	k8s	A Deployment gateway with a ClusterIP and a headless Service; gatewayExporter() points agents at it, through loadbalancing when routing by trace id
FlyOtelCollector	fly	A Fly Machine, with the config written in as a file
GkeOtelCollector	k8s	A DaemonSet agent on GKE that exports to Google Cloud

GkeOtelCollector (k8s lexicon)

GkeOtelCollector in @intentius/chant-lexicon-k8s deploys a collector DaemonSet on GKE with Workload Identity. Its config is declared through this lexicon: an otlp receiver on 4317 and 4318, batch and resourcedetection (gcp) processors, and a googlecloud exporter for the given project with a metric prefix per cluster, wired into metrics and traces pipelines. The composite renders it with collectorYaml into the config.yaml key of its ConfigMap.

Moving it onto this lexicon did not change its output. The composite’s tests compare the rendered config byte for byte with the hand-written template it used before, for several project and cluster names, and run validateCollectorConfig over it.