Connector framework

A connector is an integration with a foreign system: a WiFi controller, a DNS filter, a hypervisor API, a notification service, a threat-intelligence feed. The framework exists so that twenty of them are one shape instead of twenty special cases.

The framework is behind a feature flag and is off by default. With the flag unset, connectors.yaml is not read at all.

Three kinds, one form

Kind	Does	Example
Source	Reads events or state from a foreign system	A WiFi controller’s client list
Sink	Hands incidents, metrics or evidence outwards	A notification topic, a metrics endpoint
Responder	Executes an already approved reaction	Setting a block list entry

All three share the same three methods — Describe, Configure, Test — and the framework owns everything around them: configuration validation, secret references, health, rate limiting, backoff, the concurrency gate, the bounds and the test connection.

A connector owns exactly one thing: the conversation with its foreign system.

Rules the framework enforces

These are enforced in the framework, not trusted to the connector. That distinction is the whole point.

No connector determines its tenant. Tenant, instance name and limits come from the registry. A connector’s output is untrusted data, and the core assigns identity and trust class from configuration.

Everything is bounded. Batch size, response size (1 MiB by default) and per-call timeout. A bound that is reached sets a truncated marker, so silent cutting is not possible.

Cursors are opaque and idempotent. The framework does not interpret a cursor, and two runs over the same cursor produce the same items.

Failures are visible. Every failure carries a stable code, increments a counter, and references the retained raw records.

No connector approves anything. A responder may execute only an operation its own descriptor declares, only for its own tenant, and only after a person approved it elsewhere.

No credential is ever a configuration value. Secrets are declared separately and read through a reference at the moment of use. A plaintext secret in the configuration file is a start error, not a warning.

Health, rate limits, backoff

Every configured instance has its own health line, visible in the system view:

State	Means
unknown	Declared, never succeeded yet
ok	Last run succeeded
stale	Succeeded before, not recently
error	Failing, with a code
disabled	Switched off by an operator; keeps its cursor
unlicensed	A Pro connector without its licence feature

A declared instance that has never delivered stays visible as unknown, and a previously healthy one that stops delivering becomes stale rather than disappearing. Facts age out; they are never deleted to keep a page tidy.

The rate limit is a per-instance token bucket — requests per minute plus a burst — with at most one in-flight request per instance, so a slow foreign system cannot be overrun. A foreign 429 with Retry-After is adopted both as a bucket penalty and as a backoff floor.

Backoff is exponential with jitter, 30 seconds to 15 minutes by default, and resets only after a complete success. Jitter is only ever added, never subtracted, so the first retry cannot fall below the floor and the delay sequence stays monotonic. A connector in backoff produces no load and exactly one health entry — not an alert storm.

Versioning

Two version numbers, on purpose:

• The protocol version changes only on a breaking protocol change. The core supports the current one and the previous one.
• The connector version is the connector’s own.

A connector with an unknown kind or a too-new protocol version is rejected and reported, not ignored. Event formats grow additively only: readers before writers.

Licensing

A connector that declares a required licence feature does not register without it. It appears as unlicensed with a reason. It does not crash, and it does not half-work.

The feature comes from the current entitlement, which the appliance renews against the licence server every one to two hours. A lapsed entitlement withdraws the capability at the next renewal; an unreachable server costs nothing for 72 hours. A free connector never consults any of this. See licensing.

Testing without the foreign system

No connector test may require a live foreign system, and continuous integration needs no network. Two mechanisms make that true:

A shared conformance suite. Every connector passes the same table-driven suite or it is not registered. It checks the descriptor, that registration is safe to repeat, that a deliberately broken descriptor is rejected, that every invalid configuration is refused with a configuration code, that a plaintext or missing or world-readable secret is refused, that a disabled instance is inert and keeps its cursor, that an unlicensed one stays visible, that the test connection fails closed, the health transitions, the backoff bounds and ceiling, that no secret value reaches any operator-visible string, that Test() performs no writing request, and that the cursor does not re-deliver.

Frozen recordings. A conversation with a foreign system is recorded as JSON and replayed as an HTTP transport. The recorded failure matrix is mandatory: success, empty answer, 401, 403, 429 with Retry-After, 500, a cut payload, an unexpected shape, and a dead system. Each must produce a distinguishable code, and none of them may look like an empty data set — that is the bug class where a broken integration silently reports “nothing found”.

Recordings are anonymised deterministically: addresses inside the site networks and every globally routable address become documentation values, MAC addresses become documentation-range addresses, internal host names become example.invalid, and credential-shaped fields become a redaction marker. The mapping is stable, so a cross-reference between two responses survives — and the locally administered bit of a MAC is preserved, because the identity model reads exactly that bit to recognise a randomised address. A verification test fails if a recording still contains a real address, a real MAC or an internal domain.

Writing one

The shape, in brief:

1. Return a descriptor: identifier, name, category, version, kind, tier, the licence feature if it is a Pro connector, the emitted record kinds, a typed configuration schema, the secret list, the limits, and the foreign version the recordings were made against. An incomplete descriptor is refused, and the registry then refuses the connector.
2. Declare every configuration field in the schema with its type and bounds. The YAML validation and the operator form both come from that one declaration, so a new connector brings its own interface with no frontend change. A polling floor belongs in the schema and in the limits: floors are enforced, not recommended.
3. Never accept a credential as a configuration value. Declare it as a secret and read it through the reference at the moment of use.
4. Use the injected HTTP client, never a hand-built transport — the injected one is what a recording replaces, and it refuses redirects on purpose.
5. Return errors with a stable code, and put nothing from a foreign response into the message: a response body can contain the credential that was just sent. The framework additionally strips every configured secret value from error texts, health lines and the entire unwrap chain, and drops a wrapped cause that carried one.