NON-authoritative for deactivation (deliberately false — the T3-deadlock-safe posture).
The metadata-catalog enumerates the Record-Service record types (standard + custom) — but it is
NOT the complete gamut of this connector's emitted IOs: the contract retains sublist-access-path
children (InvoiceItem, SalesOrderItem) that are reachable as nested graph children and NEVER
appear in the record-type catalog (SublistScopeExtension). It also returns lowercase record-type
SLUGS (phonecall) while Declared standard objects are humanized (Phone Call). An authoritative
(deactivating) refresh would therefore wrongly Disable every Declared sublist child and any object
whose catalog slug doesn't byte-match the Declared name. Per the convention rule — a partial/scoped
enumeration is NEVER authoritative — absence here proves nothing, so we deactivate nothing. The
standard universe is the Declared metadata baseline; a credential only ADDS the account's customs.
The canonical integration name (e.g., "HubSpot", "Rasa.io"). Used by GetActionGeneratorConfig() and IntegrationActionExecutor to match connectors to action Config.IntegrationName.
Override in subclasses. Defaults to the class name.
NetSuite governs by CONCURRENT-request limit (5–50 by tier), not requests/sec. We cap inner concurrency conservatively below the smallest tier; the engine's adaptive controller ramps within it.
SuiteQL fetches in ORDER BY lastModifiedDate and an updated record always re-surfaces at a new,
higher modstamp — so the last batch's max IS the true high-water mark. Safe for the engine to use
the returned watermark to narrow the next incremental.
No documented RPS policy — concurrency-governed. Return null so the engine derives a conservative rate.
Whether this connector supports batched target writes (plan.md §7 aggressive batching).
Whether this connector supports creating new records in the external system.
Whether this connector supports deleting records from the external system.
Whether this connector supports reading/fetching records. Always true.
Whether this connector supports paginated listing of records.
Whether this connector supports searching/querying records with filters.
Whether this connector supports updating existing records in the external system.
Whether this connector supports idempotent upserts (create-or-update keyed
by a unique business property). Connectors override this AND Upsert to enable it.
ProtectedAppendAppends default query parameters from the IntegrationObject metadata. DefaultQueryParams is stored as a JSON object like {"key": "value"}. Automatically skips params whose key (case-insensitive) already appears in the URL to avoid duplicates with pagination params.
ProtectedapplyRuns TransformRecord, then RE-ADDS any source key the transform dropped (present in
raw, absent from the output) unless declared in ExcludedSourceKeys. This makes the
base-fetch path structurally full-record (§0 forward-compat contract): a TransformRecord
override may reshape/coerce but cannot SILENTLY drop a source key from ExternalRecord.Fields,
so the framework's custom-column capture sees everything the source returned. The default
identity transform returns raw unchanged → fast-path no-op (zero overhead, zero behavior
change for connectors that don't override TransformRecord).
ProtectedAuthenticateAuthenticate with the external system and return an auth context. Called once per FetchChanges invocation; the returned context is passed to BuildHeaders and MakeHTTPRequest for every request.
Batch-create. Default loops single-record CreateRecord, so the engine may always call the batch form.
Batch-delete. Default loops single-record DeleteRecord.
Batch-update. Default loops single-record UpdateRecord.
ProtectedBuildBuilds a CRUDResult for a record CREATE, failing LOUDLY when the external system returned no usable record ID. A 2xx response with an empty/undefined ID means the create did not durably produce a record we can track — returning Success:true there silently loses the record and causes duplicate creates on the next sync (the HubSpot-association class of bug, fixed in next commit 9f718a7e). This makes that failure explicit at the connector boundary.
ProtectedBuildBase headers WITHOUT an Authorization value — the auth header is per-request (OAuth 1.0a signs each URL+method) so it's injected in MakeHTTPRequest, not here. The base class calls BuildHeaders for its generic CRUD / pagination GETs; MakeHTTPRequest stamps the signed/bearer Authorization.
ProtectedBuildBuild the operation request body per BodyShape:
ProtectedBuildAppends pagination parameters to a URL based on the object's PaginationType. Override in subclasses to use vendor-specific parameter names.
Optionalcursor: stringOptionaleffectivePageSize: numberWhen provided by FetchPaginatedLoop, caps the requested page size to the remaining batch capacity. Subclasses should honor this to prevent overshoot.
ProtectedBuildMaps each Declared IO's Configuration.recordTypeSlug → its Declared Name (for catalog name-alignment).
Generic create: reads CreateAPIPath/Method/BodyShape/BodyKey/IDLocation from the IO row.
Generic delete: reads DeleteAPIPath/DeleteMethod/DeleteIDLocation from the IO row.
Discovers fields for a record type from the metadata-catalog JSON-schema for that type. The schema
properties map → fields; required → IsRequired; readOnly/computed markers → IsReadOnly; the
universal numeric id is the PK. Falls back to the cached Declared fields when the catalog can't
describe this type.
Discovery via the connector's READ PATH (FetchChanges), TIME-BOUNDED — the way to gather a statistically-significant sample when a single DiscoverFields sample is too small to PROVE a key. "Discovery is the sync read path with the save removed."
Loops FetchChanges as a read-only FULL fetch (WatermarkValue=null, nothing persisted), threading pagination/keyset cursors across batches, and streams every record through the data-informed field + provable-PK inference. It stops at the discovery TIME BUDGET (default 5 min), or a record cap, or source exhaustion — whichever comes first — so the provable-PK decision is made on as much real data as the budget allows. It NEVER fabricates a key: if even this larger sample yields no provable single/composite PK, the field set comes back PK-less and the object is honestly not added.
Falls back to the single-sample DiscoverFields if the read path can't run for this object (e.g. a connector whose FetchChanges needs an already-persisted IO row that doesn't exist yet).
Optionalopts: { BatchSize?: number; MaxRecords?: number; TimeBudgetMs?: number }ProtectedDiscoverStage-2 field discovery for sources WITHOUT a describe/introspection endpoint (file feeds,
undocumented JSON list endpoints): stream the source's actual records — READ-ONLY, no save,
no ack — and derive the full field set + data-informed PK/uniqueness/nullability from the
gathered statistics. The connector supplies whatever read-only fetch yields the records; this
helper turns that stream into ExternalFieldSchema[].
Why data-informed: streaming the real values lets pickPrimaryKeyFromStats pick the PK from
evidence (uniqueness/non-null statistics) COMBINED with the naming convention, rather than a
name guess alone. The PK is a SOFT key, so the pick is best-available, not strict-significance:
a confident unique+non-null column wins outright; otherwise a near-unique / convention-named
column is taken as a soft key (a PK-less object would stall CodeGen). The scan is time-bounded —
it stops on exhaustion OR opts.Discovery.TimeBudgetMs; more rows simply mean stronger claims.
Provable-only encoding into the standard flags:
IsPrimaryKey — set ONLY on the single statistics-first pick. Multiple equally-ranked unique
columns leave PK unset here (ambiguous → the pipeline's SoftPKClassifier LLM tiebreaker
decides, fed these same stats). Zero unique columns → no PK is fabricated.IsUniqueKey — set when the column was all-distinct over the scan AND uniqueness was provable
(the distinct-cap wasn't hit).AllowsNull — asserted true ONLY when a null/absent value was actually observed; otherwise
left undefined (permissive default). Never fabricates NOT NULL — critical under a time-capped
partial scan where unseen rows could still be null.The PK emitted here is SOFT (it rides additionalSchemaInfo via the persist + DDL path; it is
NEVER a hard DB key), so a wrong inference can never reject a valid row — the engine dedupes via
the record-map. IsReadOnly defaults to true (stream discovery targets read feeds); a writable
source overrides via opts.ReadOnly.
A read-only sync/async iterable of source records (the caller's fetch yields them).
Optionalopts: { Discovery?: StreamDiscoveryOptions; Pk?: PkPickOptions; ReadOnly?: boolean }OptionalDiscovery?: StreamDiscoveryOptionsTime budget / sample caps for the scan (see StreamDiscoveryOptions).
OptionalPk?: PkPickOptionsSignificance threshold + naming-rank tiebreaker (see PkPickOptions).
OptionalReadOnly?: booleanWhether discovered fields are read-only. Default true.
Discovers objects by UNIONING the Declared baseline with the live metadata-catalog (case-2
auth-gated discovery). The Declared metadata (200+ standard record types + the sublist-access-path
children that the catalog never lists) is the floor — it is ALWAYS surfaced so a refresh can't drop
it. The catalog's record-type SLUGS (phonecall) are name-aligned back to the Declared name via the
cached Configuration.recordTypeSlug so a slug never duplicates an existing Declared object; an
unknown slug passes through as a NEW custom record. Net: standard universe always present + tenant
customs added. Non-authoritative (DiscoveryIsAuthoritative=false), so no object is ever deactivated.
ProtectedDiscoveryDISCOVERY-ONLY: last-resort SAFETY bound on the recursion depth for multi-level template chains (a
parent that is itself a template-var child), so a malformed metadata cycle cannot loop unboundedly.
This is only the fallback when the consumer sets no discoverySampleMaxDepth in Configuration and
no env override is present — the consumer decides the real value. A recursion cap MUST have some
bound, so a conservative default remains here; override the getter to change the fallback.
ProtectedDiscoveryREST override (§sample-discover): a SINGLE-template-var CHILD is sampled with the recursive, record-constrained StreamRecordsForDiscovery. Flat objects fall back to the generic FetchChanges loop; MULTI-var (composition) children are deferred — StreamRecordsForDiscovery adjourns them (declared-only until first sync) rather than fire malformed URLs.
ProtectedExcludedSource keys a connector DELIBERATELY removes in its TransformRecord override (vendor noise, a flattened parent blob, a nested child collection emitted as its own object). These are the only sanctioned drops — excluded from the applyTransformPreservingKeys re-add. Default: none. Override per-object to declare intentional removals.
ProtectedExtractBest-effort error message extraction from a vendor response. Override for vendor-specific shapes.
ProtectedExtractExtract the new record's external ID from a create response per IDLocation.
ProtectedExtractOffset pagination: hasMore flag + offset advance (or the next-link href when present).
Parse NetSuite's 429 Retry-After (seconds) into ms; concurrency limit errors carry no precise hint.
Fetches records from the external REST API using metadata-driven configuration. Handles both flat endpoints and template-variable (per-parent) endpoints.
Returns the ActionGeneratorConfig for this connector, combining the integration name, category, icon, and objects into a ready-to-use configuration for ActionMetadataGenerator.Generate().
Override in subclasses to customize the config (e.g., icon, category). Returns null by default if GetIntegrationObjects() returns empty.
ProtectedGetBase URL = host root; the IO's APIPath already carries the full /services/rest/... path.
ProtectedGetGets IntegrationObjectField records from the engine's cache for a given object ID. Returns only active fields sorted by Sequence.
ProtectedGetGets an IntegrationObject from the engine's cache by integration ID and object name. Throws if not found.
ProtectedGetThe Declared (cached) baseline objects — the floor DiscoverObjects always surfaces. Seam over the base class's cache read so the union logic is unit-testable without a DB-backed engine.
Returns a proposed default configuration for quick setup. Override in subclasses to provide connector-specific defaults including schema name, objects to sync, and field mappings. Returns null by default (no quick setup available).
Returns suggested default field mappings for an external object to MJ entity. Override in subclasses to provide intelligent defaults.
Name of the external object
Name of the target MJ entity
Array of default field mappings (empty by default)
Returns the integration objects and their fields that this connector supports, for use by the ActionMetadataGenerator. This is static metadata that does NOT require a live connection — it describes the connector's known object model.
Override in subclasses to provide connector-specific objects/fields. Returns an empty array by default (no action generation available).
Generic get-one: hits APIPath/{ID} via GET. Override if API uses non-standard get shape.
Builds SourceSchemaInfo from IntegrationEngineBase cached metadata. Provides richer metadata than the base class implementation, including FK relationships derived from RelatedIntegrationObjectID.
Lists records with cursor-based pagination.
Override in subclasses that support paginated listing.
Check SupportsListing before calling.
ProtectedMakeExecute an HTTP request. The concrete connector owns the transport layer (fetch, axios, got, etc.).
Optionalbody: unknownProtectedNormalizeStrips the NetSuite HATEOAS envelope: { items: [...] } → the items array; bare object → [object].
Type-driven post-processing hook (plan.md §10): a connector may normalize/enforce a record's
values to the resolved column formats AFTER transform/normalize and BEFORE write. Default
returns the record unchanged. (Named for this system — NOT MCP, not take.) The engine ALSO
applies target-type constraint enforcement; this is the connector-side complement.
Searches for records matching the given filters.
Override in subclasses that support search/query operations.
Check SupportsSearch before calling.
For watermark-less objects, NetSuite's universal numeric id is a stable monotonic ordering key
usable for keyset/seek resume.
ProtectedSubstituteSubstitute the ExternalID into a URL path template at runtime. Path templates may use {ID}, {id}, or {ExternalID} as the placeholder; for symmetry with FetchChanges template-var handling, any unsubstituted {var} in the template is left in place (caller's responsibility to ensure consistency).
For IDLocation='body' or 'header' the ID is not substituted into the path (caller must handle separately); we just return the raw path.
ProtectedTemplatePer-call cap on how many PARENTS a parent-iterated object processes before yielding a resumable batch (the engine loops on HasMore). Kept conservative so a single FetchChanges call fits the engine's op-timeout even at a slow initial adaptive rate; the AIMD bucket ramps up over batches. Override in a concrete connector if a vendor tolerates larger per-call parent runs.
Validates credentials + host by running a trivial SuiteQL probe (no record dependency).
ProtectedTransformOptional per-record CUSTOMIZATION hook called between NormalizeResponse (vendor envelope-stripping) and ToExternalRecord (composite-PK assembly).
Distinct from NormalizeResponse — that strips the vendor's response envelope to expose individual records. This hook is for vendor-specific record-level customization the standard pipeline shouldn't carry: nested-field flattening, empty-string→null coercion for date columns, computed fields, removing vendor metadata blobs (e.g. Salesforce 'attributes'), etc.
Default is identity (no transform). Override only when a concrete connector needs vendor-specific shape changes.
Generic update: reads UpdateAPIPath/Method/BodyShape/BodyKey/IDLocation from the IO row.
Upserts a record — a single idempotent create-or-update keyed by a unique
business property (e.g. email), eliminating the search-then-create race window.
Override in subclasses whose external system exposes a keyed upsert primitive.
Check SupportsUpsert before calling.
Abstract base class for REST API integration connectors.
Implements the generic REST sync pattern: reads IntegrationObject/Field metadata from the MJ database, handles pagination, template variable resolution (per-parent iteration), and converts raw API responses to ExternalRecord format.
Concrete connectors (YourMembership, Salesforce, HubSpot, etc.) extend this class and implement only auth, HTTP transport, and response normalization.