Class RhythmConnector

Appends default query parameters from the IntegrationObject metadata. DefaultQueryParams is stored as a JSON object like {"key": "value"}. Automatically skips params whose key (case-insensitive) already appears in the URL to avoid duplicates with pagination params.

Runs TransformRecord, then RE-ADDS any source key the transform dropped (present in raw, absent from the output) unless declared in ExcludedSourceKeys. This makes the base-fetch path structurally full-record (§0 forward-compat contract): a TransformRecord override may reshape/coerce but cannot SILENTLY drop a source key from ExternalRecord.Fields, so the framework's custom-column capture sees everything the source returned. The default identity transform returns raw unchanged → fast-path no-op (zero overhead, zero behavior change for connectors that don't override TransformRecord).

Builds a CRUDResult for a record CREATE, failing LOUDLY when the external system returned no usable record ID. A 2xx response with an empty/undefined ID means the create did not durably produce a record we can track — returning Success:true there silently loses the record and causes duplicate creates on the next sync (the HubSpot-association class of bug, fixed in next commit 9f718a7e). This makes that failure explicit at the connector boundary.

Build the operation request body per BodyShape:

• 'flat' → body = attributes verbatim
• 'wrapped' → body = { [BodyKey]: attributes }
• 'literal' → connector should have overridden the operation; fall back to flat as safety net
• null → default to flat (most common shape)

Appends pagination parameters to a URL based on the object's PaginationType. Override in subclasses to use vendor-specific parameter names.

Stage-2 field discovery for sources WITHOUT a describe/introspection endpoint (file feeds, undocumented JSON list endpoints): stream the source's actual records — READ-ONLY, no save, no ack — and derive the full field set + data-informed PK/uniqueness/nullability from the gathered statistics. The connector supplies whatever read-only fetch yields the records; this helper turns that stream into ExternalFieldSchema[].

Why data-informed: streaming the real values lets pickPrimaryKeyFromStats pick the PK from evidence (uniqueness/non-null statistics) COMBINED with the naming convention, rather than a name guess alone. The PK is a SOFT key, so the pick is best-available, not strict-significance: a confident unique+non-null column wins outright; otherwise a near-unique / convention-named column is taken as a soft key (a PK-less object would stall CodeGen). The scan is time-bounded — it stops on exhaustion OR opts.Discovery.TimeBudgetMs; more rows simply mean stronger claims.

Provable-only encoding into the standard flags:

• IsPrimaryKey — set ONLY on the single statistics-first pick. Multiple equally-ranked unique columns leave PK unset here (ambiguous → the pipeline's SoftPKClassifier LLM tiebreaker decides, fed these same stats). Zero unique columns → no PK is fabricated.
• IsUniqueKey — set when the column was all-distinct over the scan AND uniqueness was provable (the distinct-cap wasn't hit).
• AllowsNull — asserted true ONLY when a null/absent value was actually observed; otherwise left undefined (permissive default). Never fabricates NOT NULL — critical under a time-capped partial scan where unseen rows could still be null.

The PK emitted here is SOFT (it rides additionalSchemaInfo via the persist + DDL path; it is NEVER a hard DB key), so a wrong inference can never reject a valid row — the engine dedupes via the record-map. IsReadOnly defaults to true (stream discovery targets read feeds); a writable source overrides via opts.ReadOnly.

DISCOVERY-ONLY: last-resort SAFETY bound on the recursion depth for multi-level template chains (a parent that is itself a template-var child), so a malformed metadata cycle cannot loop unboundedly. This is only the fallback when the consumer sets no discoverySampleMaxDepth in Configuration and no env override is present — the consumer decides the real value. A recursion cap MUST have some bound, so a conservative default remains here; override the getter to change the fallback.

REST override (§sample-discover): a SINGLE-template-var CHILD is sampled with the recursive, record-constrained StreamRecordsForDiscovery. Flat objects fall back to the generic FetchChanges loop; MULTI-var (composition) children are deferred — StreamRecordsForDiscovery adjourns them (declared-only until first sync) rather than fire malformed URLs.

Source keys a connector DELIBERATELY removes in its TransformRecord override (vendor noise, a flattened parent blob, a nested child collection emitted as its own object). These are the only sanctioned drops — excluded from the applyTransformPreservingKeys re-add. Default: none. Override per-object to declare intentional removals.

Best-effort error message extraction from a vendor response. Override for vendor-specific shapes.

Extract the new record's external ID from a create response per IDLocation.

Gets IntegrationObjectField records from the engine's cache for a given object ID. Returns only active fields sorted by Sequence.

Gets an IntegrationObject from the engine's cache by integration ID and object name. Throws if not found.

Substitute the ExternalID into a URL path template at runtime. Path templates may use {ID}, {id}, or {ExternalID} as the placeholder; for symmetry with FetchChanges template-var handling, any unsubstituted {var} in the template is left in place (caller's responsibility to ensure consistency).

For IDLocation='body' or 'header' the ID is not substituted into the path (caller must handle separately); we just return the raw path.

Per-call cap on how many PARENTS a parent-iterated object processes before yielding a resumable batch (the engine loops on HasMore). Kept conservative so a single FetchChanges call fits the engine's op-timeout even at a slow initial adaptive rate; the AIMD bucket ramps up over batches. Override in a concrete connector if a vendor tolerates larger per-call parent runs.

Optional per-record CUSTOMIZATION hook called between NormalizeResponse (vendor envelope-stripping) and ToExternalRecord (composite-PK assembly).

Distinct from NormalizeResponse — that strips the vendor's response envelope to expose individual records. This hook is for vendor-specific record-level customization the standard pipeline shouldn't carry: nested-field flattening, empty-string→null coercion for date columns, computed fields, removing vendor metadata blobs (e.g. Salesforce 'attributes'), etc.

Default is identity (no transform). Override only when a concrete connector needs vendor-specific shape changes.