Skip to content
MemberJunction

MemberJunction Integration Engine

A pluggable, metadata-driven integration framework for syncing data between external systems (CRMs, AMSes, file feeds) and MemberJunction entities.

Vendor connectors now live in MemberJunction/Integrations and install on demand as Open Apps — the core install no longer carries the full IntegrationObject/IntegrationObjectField catalog for every vendor.

Terminal window
mj app install https://github.com/MemberJunction/Integrations/CRM/HubSpot

Each connector is its own Open App + npm package @memberjunction/connector-<name>: installing one npm installs only that connector’s package and runs that connector’s Skyway seed migrations (generated from its metadata/ via mj sync push), which seed that connector’s Integration + IO/IOF + Actions into the shared __mj schema. Connectors are fully decoupled — installing HubSpot pulls no other connector. This package (@memberjunction/integration-engine et al.) retains the framework (engine, schema-builder, pk-classifier, actions, the Integration/IO/IOF tables, runtime discovery, credential types) — connectors are extensions on top of it, consumed as peer dependencies. Seeded IO/IOF are not required up front: IntegrationConnectorCreationPipeline also discovers them at credential-setup time.

Architecture

Section titled “Architecture”
packages/Integration/
├── engine/          @memberjunction/integration-engine     — Core orchestration
├── connectors/      @memberjunction/integration-connectors — Connector implementations
├── actions/         @memberjunction/integration-actions    — MJ Actions for scheduling
├── ui-types/        @memberjunction/integration-ui-types   — Angular view model types
└── e2e/                                                    — End-to-end test scripts

Quick Start

Section titled “Quick Start”

Run a sync from code

Section titled “Run a sync from code”
import { IntegrationOrchestrator } from '@memberjunction/integration-engine';


const orchestrator = new IntegrationOrchestrator();
const result = await orchestrator.RunSync(
    companyIntegrationID,
    contextUser,
    'Manual',
    (progress) => console.log(`${progress.PercentComplete}% complete`),
    (notification) => sendEmail(notification.Subject, notification.Body)
);


console.log(`Processed: ${result.RecordsProcessed}, Created: ${result.RecordsCreated}`);

Run a sync via MJ Actions

Section titled “Run a sync via MJ Actions”

Create a MJ: Actions record with Name = Run Integration Sync and register it in the scheduler. The action accepts:

ParameterTypeRequiredDescription
CompanyIntegrationIDInputYesID of the CompanyIntegration to sync
TriggerTypeInputNoManual, Scheduled, or Webhook (default: Manual)

Data Model

Section titled “Data Model”

The integration engine reads from and writes to these MJ entities:

EntityPurpose
MJ: IntegrationsIntegration type definitions (HubSpot, Salesforce, etc.)
MJ: Integration Source TypesAvailable connector drivers
MJ: Company IntegrationsInstance of an integration for a company
MJ: Company Integration Entity MapsMaps external objects → MJ entities
MJ: Company Integration Field MapsMaps external fields → MJ fields with transforms
MJ: Company Integration RunsAudit trail of each sync execution
MJ: Company Integration Run DetailsPer-entity stats for a run
MJ: Company Integration Record MapsTracks external ID ↔ MJ record ID mapping
MJ: Company Integration Sync WatermarksIncremental sync state per entity map

Key Concepts

Section titled “Key Concepts”

Watermarks — Each entity map stores a watermark (timestamp, cursor, or version token) that marks the last successfully synced position. On the next run, the connector fetches only records modified after the watermark.

Field Transforms — Each field map can have a TransformPipeline JSON array that chains transform steps: direct, regex, split, combine, lookup, format, coerce, substring, custom.

Record Matching — Before writing records, the match engine checks: (1) key field matching via SQL filter, (2) CompanyIntegrationRecordMap lookup, (3) conflict resolution strategy.

Notifications — After a sync completes or fails, the orchestrator emits a SyncNotification with a pre-formatted subject and body suitable for email delivery.

Integration Actions (Auto-Generated CRUD)

Section titled “Integration Actions (Auto-Generated CRUD)”

Each connector can automatically generate a full set of Get, Create, Update, Delete, Search, and List actions for every object it supports. These actions are discoverable by AI agents, workflows, and low-code tools through the standard MJ Actions API.

See INTEGRATION_ACTIONS.md for the complete architecture guide, current connector inventory, and instructions for adding new connectors.

Packages

Section titled “Packages”