MemberJunction Server Scaling & Pluggable Broadcast Architecture

Executive Summary

MemberJunction’s API server (@memberjunction/server / MJAPI) powers interactive browser clients (MJExplorer), AI agent orchestration, real-time status streaming, and enterprise integrations. As deployments grow from 100 concurrent users to 5,000–50,000+ concurrent active users, four structural bottlenecks previously constrained single-node and multi-replica throughput:

1. $O(U \times E)$ Subscription Fan-Out & Unfiltered Write Telemetry Broadcasts:
   • Every BaseEntity save or delete—including high-frequency internal operational writes (MJ: AI Prompt Runs, MJ: AI Agent Run Steps, MJ: Audit Logs, MJ: Action Execution Logs)—triggered a global CACHE_INVALIDATION event across every connected WebSocket.
   • For each event, CacheInvalidationFilter and ShapeCacheInvalidationEvent evaluated entity.GetUserPermisions(user) and ReadIsRowFiltered(entity, user) independently for all $U$ connected users ($O(U)$ per event), even when 5,000 users shared only 4 distinct role combinations.
   • Similarly, PushStatusResolver.statusUpdates subscribed all $U$ clients to a single global PUSH_STATUS_UPDATES topic, running StatusUpdatesFilter $U$ times per AI streaming token.
2. Per-Field GraphQL Middleware & Per-Request Context Allocation Overhead:
   • Registering VariablesLoggingMiddleware in type-graphql’s globalMiddlewares wrapped all ~45,000 object/field resolvers in an async middleware chain. A RunView returning 1,000 rows × 30 fields allocated 30,000 Promises per query even when logVariables was disabled.
   • Every HTTP request cloned all 19 AllMetadataArrays (~15,000 object references per provider), eagerly allocated RxJS Subject/BehaviorSubject transaction queues on SQLServerDataProvider (even though >99% of requests are non-transactional), re-decoded JWT tokens via jwt.decode(), and executed uncached RunView queries during APIKeyEngine.ValidateAPIKey.
3. Database Connection Pool Stampedes on Identical Concurrent Reads:
   • Simultaneous identical RunView queries (e.g., 500 users opening a dashboard at 9:00 AM) each acquired a separate SQL connection and executed identical SQL concurrently, exhausting the 50-connection pool.
   • Unbounded Promise.all inside GenericDatabaseProvider.InternalRunViews allowed a single batched request with 20 views to monopolize 20 pool connections at once.
4. Coupled Interactive API & Background Worker Loops:
   • Every MJAPI replica unconditionally started background polling daemons (TaskGraphDispatcher, ScheduledJobsService, IntegrationSyncWorkerService, WorkQueueHost, SessionJanitor, OrphanDetailReconciler), causing multi-replica poll contention and event-loop latency spikes on interactive HTTP/WebSocket nodes.

---

The Four Pillars of the Scalability Architecture

flowchart TB
    subgraph Clients["Connected Clients (1,000 – 50,000+ Browsers)"]
        Browser1["MJExplorer Tab (Role: UI)"]
        Browser2["MJExplorer Tab (Role: Developer)"]
        Browser3["Sleeping / Reconnecting Tab"]
    end

    subgraph Pillar1["Pillar 1: Pluggable Cache Invalidation Broadcast & O(1) Session PubSub"]
        BaseProvider["BaseCacheInvalidationBroadcastProvider<br/>• Smart Telemetry Entity Filter<br/>• Role-Fingerprint O(Roles) Memoizer<br/>• Monotonic Sequence Ring Buffer"]
        DefaultProvider["SequencedRoleBucketBroadcastProvider<br/>(Default: Topology 1 + Topology 2)"]
        CloudProviders["Topology 3 Cloud Multiplexers<br/>• AzureWebPubSubBroadcastProvider<br/>• AWSIoTCoreBroadcastProvider<br/>• GCPPubSubBroadcastProvider"]
        SessionPubSub["O(1) Session-Scoped PushStatus<br/>PUSH_STATUS_UPDATES:session:<id>"]
        BaseProvider --> DefaultProvider
        BaseProvider --> CloudProviders
    end

    subgraph Pillar2["Pillar 2: Zero-Overhead Request & GraphQL Pipeline"]
        RootMw["Root-Only VariablesLoggingMiddleware<br/>(0 wrappers on child ObjectType fields)"]
        ProtoShell["O(1) Prototype Metadata Shell<br/>Object.create(shared)"]
        LazyTx["Lazy RxJS Transaction Queue<br/>(Allocated only on BeginTransaction)"]
        AuthCaching["AuthCache JWT Payload Cache +<br/>APIKeyEngine TTL Validation Cache"]
    end

    subgraph Pillar3["Pillar 3: Database Pool Protection"]
        SingleFlight["Single-Flight In-Flight RunView Coalescing<br/>(Deduplicates concurrent identical SQL reads)"]
        BoundedBatch["Bounded InternalRunViews Concurrency<br/>(Default max 8 concurrent SQL legs/batch)"]
    end

    subgraph Pillar4["Pillar 4: Horizontal Replica Role Separation"]
        ApiNodes["MJ_SERVER_ROLE=api<br/>Stateless HTTP + WebSocket Fleet"]
        WorkerNodes["MJ_SERVER_ROLE=worker<br/>Dedicated Background Daemons"]
    end

    Clients <-->|"HTTP (X-MJ-Cache-Seq) & WS"| ApiNodes
    Clients <-->|"Optional Topology 3 WS"| CloudProviders
    ApiNodes --> Pillar1
    ApiNodes --> Pillar2
    ApiNodes --> Pillar3
    WorkerNodes --> Pillar1
    WorkerNodes --> Pillar3

---

Throughput & Scalability Math

Metric	Previous Architecture	New Architecture (Topology 1+2 Default)	New Architecture (Topology 3 Cloud Multiplexer)
Permission & RLS checks per entity save ($U=5,000$ users, $R=4$ role combos)	$5,000 \times 2 = 10,000$ evaluations	$4 \times 2 = 8$ evaluations ($1,250\times$ reduction)	$4 \times 2 = 8$ evaluations ($1,250\times$ reduction)
Broadcasts for telemetry tables (AIPromptRun, AuditLog, etc.)	Broadcasts every row to all $U$ clients	Filtered at source ($0$ broadcasts)	Filtered at source ($0$ broadcasts)
PushStatus filter evaluations per AI token ($U=5,000$ users)	$5,000$ filter invocations / token	$1$ targeted session invocation ($O(1)$)	$1$ targeted session invocation ($O(1)$)
GraphQL middleware Promises per 1,000-row × 30-field RunView	$30,000$ Promises	$0$ Promises (root-only when enabled)	$0$ Promises
Metadata array copies per request	$\sim 30,000$ reference copies	$0$ copies (Object.create(shared))	$0$ copies
500 concurrent identical dashboard RunViews	$500$ SQL queries (pool exhaustion)	$1$ SQL query (single-flight coalesced)	$1$ SQL query
Reconnect / Tab-wake catch-up cost	Full engine/dataset reload ($10\text{–}50$ SQL queries)	$<1\text{ ms}$ RAM ring-buffer delta replay	$<1\text{ ms}$ RAM ring-buffer delta replay
Practical Concurrent Active Users	$\sim 100\text{–}250$ users	$5,000\text{–}10,000+$ users (2–4 API replicas)	$50,000+$ users (offloaded WS fan-out)

---

Pillar 1: Pluggable Cache Invalidation Broadcast Hierarchy & $O(1)$ Session PubSub

1.1 BaseCacheInvalidationBroadcastProvider

Registered and resolved via MemberJunction’s MJGlobal.Instance.ClassFactory (@RegisterClass(BaseCacheInvalidationBroadcastProvider, key)), BaseCacheInvalidationBroadcastProvider provides the foundational intelligence shared by all broadcast topologies:

1. Smart Cacheable Entity Filter (ShouldBroadcastEntity(entityName: string): boolean):

   • High-frequency write-only operational and telemetry entities (MJ: AI Prompt Runs, MJ: AI Agent Run Steps, MJ: AI Agent Runs, MJ: Action Execution Logs, MJ: Audit Logs, MJ: Record Changes, MJ: User Record Logs, MJ: Error Logs, MJ: Query Execution Logs, MJ: Communication Logs) are never cached by browser BaseEngine instances or metadata datasets.
   • By default, ShouldBroadcastEntity suppresses client broadcasts for these operational tables while allowing operators to override via cacheSettings.alwaysBroadcastEntities and cacheSettings.excludedBroadcastEntities.
   • Any entity that is part of the MJ_Metadata dataset or explicitly allowlisted is always broadcast.

2. Role-Fingerprint Permission & Event Shaping Memoizer ($O(\text{Roles})$ instead of $O(\text{Users})$):

   • Computes a canonical RoleFingerprint for each UserInfo / RoleHolder: sorted, lowercased, trimmed RoleIDs joined by ',' (e.g. 'role-a,role-b'), cached in a WeakMap<object, string> so a connected user’s role fingerprint is computed at most once per session.
   • When a CacheInvalidationPayload is evaluated, the provider attaches a per-event memoization map Map<string, RoleBucketEvaluation> keyed by RoleFingerprint.
   • The first subscriber with a given RoleFingerprint computes CanRead (entity.GetUserPermisions(user).CanRead), WithholdKey (ReadIsRowFiltered(entity, user)), and the shaped CacheInvalidationNotification object. Every subsequent subscriber sharing that RoleFingerprint receives the pre-computed verdict and shaped notification in $O(1)$ time with zero object allocation!

3. Monotonic Sequence Counter & Bounded RAM Ring Buffer (Topology 1):

   • Every broadcast invalidation event is stamped with a strictly increasing integer Sequence (1, 2, 3, ...) and stored in a bounded circular ring buffer (default 1,000 events, configurable via cacheSettings.invalidationRingBufferSize).
   • Exposes GetCurrentSequence(): number, GetServerProcessID(): string, and GetChangesSince(sinceSequence: number, user: UserInfo): CacheInvalidationCatchUpResult:
     • status: 'current': sinceSequence === currentSequence — no events missed.
     • status: 'delta': sinceSequence < currentSequence and covered by the ring buffer — returns only the events in (sinceSequence .. currentSequence] that the user’s RoleFingerprint is permitted to read, already shaped for row-level security!
     • status: 'stale_full_refresh_required': sinceSequence < oldestSequenceInBuffer or sinceSequence > currentSequence (e.g., server restarted or tab slept beyond ring buffer capacity) — instructs the client to run its standard full staleness validation.

sequenceDiagram
    participant Client as Browser (GraphQLDataProvider)
    participant API as MJAPI (Express + Apollo)
    participant Provider as SequencedRoleBucketBroadcastProvider
    participant RingBuffer as RAM Ring Buffer (Last 1,000 Events)

    Note over Client,API: Normal Real-Time Flow (Topology 2)
    API->>Provider: DispatchInvalidation(payload)
    Provider->>Provider: Check ShouldBroadcastEntity(entityName)
    Provider->>RingBuffer: Assign Sequence #104 & Store
    Provider-->>Client: WS Push: cacheInvalidation { Sequence: 104, ... } (O(Roles) memoized)
    Client->>Client: Update _lastCacheInvalidationSequence = 104

    Note over Client,API: HTTP Piggyback Header & Reconnect Catch-Up (Topology 1)
    Client->>API: POST /graphql (Any Query/Mutation)
    API-->>Client: HTTP 200 + Header X-MJ-Cache-Seq: 108, X-MJ-Server-ID: <uuid>
    Client->>Client: Detects gap (104 -> 108) or WS Reconnect
    Client->>API: Query GetCacheInvalidationsSince(sinceSequence: 104, serverProcessID: <uuid>)
    API->>Provider: GetChangesSince(104, user)
    Provider->>RingBuffer: Slice (104..108] + Filter/Shape via Role-Bucket Memoizer
    API-->>Client: { Status: 'delta', CurrentSequence: 108, Events: [#105..#108] }
    Client->>Client: Replay missed remote-invalidate events in <1ms!

1.2 Default Provider: SequencedRoleBucketBroadcastProvider ('SequencedRoleBucket')

Registered with @RegisterClass(BaseCacheInvalidationBroadcastProvider, 'SequencedRoleBucket'), this is the default provider out of the box (requiring zero external cloud services):

• Real-Time Push (Topology 2): Publishes SequencedCacheInvalidationPayload (carrying its pre-warmed role-bucket memoization cache) to PubSubManager. CacheInvalidationFilter and ShapeCacheInvalidationEvent automatically consult the payload’s role-bucket cache, reducing CPU and GC pressure by $>99%$ on multi-thousand-user servers.
• HTTP Sequence Header & Delta Catch-Up (Topology 1):
  • Express middleware attaches X-MJ-Cache-Seq and X-MJ-Server-ID (Access-Control-Expose-Headers included) to every GraphQL HTTP response.
  • CacheInvalidationResolver exposes @Query(() => CacheInvalidationCatchUpOutput) GetCacheInvalidationsSince(sinceSequence: Int!, serverProcessID?: String).
  • GraphQLDataProvider tracks _lastCacheInvalidationSequence and _lastCacheServerProcessId, automatically invoking CatchUpCacheInvalidations() on WebSocket reconnect (SocketReconnected$) or when an HTTP response’s X-MJ-Cache-Seq header advances past the client’s known sequence.

1.3 Topology 3 Cloud Multiplexer Providers (50,000+ Concurrent Users)

When scaling beyond 10,000–50,000 concurrent WebSockets, terminating idle WebSockets on managed cloud pub/sub multiplexers offloads socket heartbeats and TLS state from Node.js API containers. Because BaseCacheInvalidationBroadcastProvider groups permissions by RoleFingerprint, a deployment with 50,000 users and 8 role combinations publishes at most 8 messages per entity change to the cloud multiplexer, which then fans out to all subscribed browsers on that role topic (role:<fingerprint>):

1. AzureWebPubSubBroadcastProvider (@RegisterClass(BaseCacheInvalidationBroadcastProvider, 'AzureWebPubSub')):
   • Publishes role-shaped payloads to Azure Web PubSub group mj.role.<fingerprintHash> via the Azure Web PubSub REST API (POST /api/hubs/{hub}/groups/{group}/:send) using HMAC-SHA256 JWT authentication (or pluggable transport adapter).
   • NegotiateClientConnection(user) mints a client access URL (wss://<host>/client/hubs/<hub>?access_token=<jwt>) scoped to webpubsub.joinLeaveGroup.mj.role.<fingerprintHash>.
2. AWSIoTCoreBroadcastProvider (@RegisterClass(BaseCacheInvalidationBroadcastProvider, 'AWSIoTCore')):
   • Publishes role-shaped payloads to AWS IoT Core Data Plane MQTT topic mj/cache-invalidation/role/<fingerprintHash> via AWS SigV4 signed HTTPS POST (POST /topics/mj%2Fcache-invalidation%2Frole%2F<fingerprintHash>?qos=1).
   • NegotiateClientConnection(user) returns a SigV4 pre-signed wss:// MQTT-over-WebSocket URL and the authorized topic name.
3. GCPPubSubBroadcastProvider (@RegisterClass(BaseCacheInvalidationBroadcastProvider, 'GCPPubSub')):
   • Publishes role-shaped payloads to Google Cloud Pub/Sub (POST https://pubsub.googleapis.com/v1/projects/{projectId}/topics/{topicPrefix}-{fingerprintHash}:publish) with role-fingerprint attributes.
   • NegotiateClientConnection(user) returns the subscription topic/filter metadata for the user’s RoleFingerprint.

Clients can query NegotiateCacheBroadcastConnection on CacheInvalidationResolver at any time to discover the active broadcast topology and obtain signed connection credentials.

1.4 $O(1)$ Dynamic Session Topic Routing in PushStatusResolver

Previously, PushStatusResolver.statusUpdates subscribed every client to PUSH_STATUS_UPDATES, causing graphql-subscriptions’s EventEmitter to wake all $U$ iterators on every AI streaming token.

• GetSessionStatusTopic(sessionId) returns PUSH_STATUS_UPDATES:session:${sessionId}.
• PushStatusResolver.statusUpdates subscribes to [GetSessionStatusTopic(args.sessionId), PUSH_STATUS_UPDATES_TOPIC].
• PubSubManager routes any publish on PUSH_STATUS_UPDATES_TOPIC carrying a concrete sessionId (sessionId !== '*') directly to GetSessionStatusTopic(sessionId), waking only the 1 target session ($O(1)$) while wildcard sessionId: '*' broadcasts continue to publish on PUSH_STATUS_UPDATES_TOPIC.
• ShouldReplicateStatusUpdate memoizes parsed JSON envelopes via a WeakMap so JSON.parse(message) is never repeated for the same payload.

---

Pillar 2: Eliminating Per-Field & Per-Request GraphQL Execution Overhead

2.1 Root-Only VariablesLoggingMiddleware

type-graphql’s globalMiddlewares attaches every middleware function to every @Field() on every @ObjectType() in the schema (~45,000 fields). Because VariablesLoggingMiddleware only inspects root operations (info.path.prev === undefined) when loggingSettings.graphql.logVariables === true, attaching it globally caused 30,000 redundant async middleware invocations on a 1,000-row × 30-field RunView response.

• buildSchemaSync no longer registers VariablesLoggingMiddleware in globalMiddlewares.
• Instead, ApplyRootVariablesLoggingMiddleware(schema) wraps only the root fields on schema.getQueryType(), schema.getMutationType(), and schema.getSubscriptionType() (and only when logVariables is enabled or via a lightweight root-only wrapper). Child @ObjectType row fields have zero middleware wrappers.

2.2 Zero-Copy CreateSharedMetadataShell (Object.create(shared))

In ProviderBase.CreateSharedMetadataShell(shared):

• Instead of allocating a new AllMetadata and shallow-copying all 19 AllMetadataArrays ([...shared[m.key]], copying ~15,000 pointers per provider), CreateSharedMetadataShell creates a prototype-backed shell via Object.create(shared) as AllMetadata and assigns shell.CurrentUser = this.CurrentUser as an own property.
• Reading shell.Entities, shell.Applications, etc. resolves in $O(1)$ through the prototype chain with zero array copying, while assigning shell.Entities = ... or shell.CurrentUser = ... writes to the per-request shell instance without mutating shared.

2.3 Lazy RxJS Transaction Queue in SQLServerDataProvider

In SQLServerDataProvider:

• Previously, every new SQLServerDataProvider() eagerly allocated _sqlQueue$ = new Subject<SQLQueueItem>() and _transactionState$ = new BehaviorSubject<boolean>(false).
• Now, _sqlQueue$ and _transactionState$ are allocated lazily on first access or when BeginTransaction() / initializeQueueProcessor() is called. For the $>99%$ of HTTP requests that never start a transaction, IsTransactionActive returns false and Dispose() completes in $O(1)$ without allocating or tearing down RxJS subjects.

2.4 AuthCache Decoded JWT Payload Caching & APIKeyEngine Validation Caching

• AuthCache (packages/MJServer/src/cache.ts & context.ts):
  • Stores the verified jwt.JwtPayload directly in AuthCache (AuthCache.get(token)). On subsequent requests with the same warm bearer token, GetUserPayload retrieves the verified JwtPayload from AuthCache without re-running jwt.decode(token) (base64 decode + JSON.parse), while still resolving the live UserInfo from UserCache.Instance.
• APIKeyEngine.ValidateAPIKey (packages/APIKeys/Engine/src/APIKeyEngine.ts):
  • Adds a short-TTL (default 60s, configurable via validationCacheTTLMs) in-memory validation cache for ValidateKeyByHash and active UserInfo resolution, automatically invalidated whenever MJ: API Keys, MJ: API Key Scopes, MJ: API Key Applications, or MJ: Users records are saved or deleted via MJGlobal entity events (and on RevokeAPIKey / ClearCache).

---

Pillar 3: Server-Side Single-Flight Read Coalescing & Bounded InternalRunViews Concurrency

3.1 Single-Flight In-Flight RunView Coalescing

In GenericDatabaseProvider.RunViewCore:

• When multiple concurrent requests execute an identical read-only query (!this.InTransaction, !params.SaveViewResults, ResultType !== 'entity_object' or raw row fetch before entity hydration), GenericDatabaseProvider computes a deterministic coalescing key from:
  • Provider connection identity (InstanceConnectionString / core schema)
  • EntityName / ViewID / ViewName
  • ExtraFilter, OrderBy, Fields, UserSearchString, ExcludeUserViewRunID, ExcludeDataFromAllPriorViewRuns, OverrideExcludeFilter
  • MaxRows, StartRow, IgnoreMaxRows, AfterKey, ResultType, DataSource, Aggregates
  • Effective RLS / User scope key (rlsWhereClause + user’s RoleFingerprint + User.ID when RLS or view where clause is user-dependent)
• If an identical query is currently in flight, subsequent callers await the same in-flight Promise and receive an isolated clone of the result (Results: result.Results.map(r => ({ ...r }))), preventing any caller mutation from affecting peers.
• In finally, the key is immediately removed from _inFlightRunViews.
• Zero staleness risk: Results are never retained after the SQL query settles; only queries executing at the exact same millisecond share the database round-trip.

3.2 Bounded Concurrency in GenericDatabaseProvider.InternalRunViews

In GenericDatabaseProvider.InternalRunViews:

• Replaces unbounded Promise.all(params.map(...)) with a bounded concurrency limiter (default MaxConcurrentViewsPerBatch = 8, configurable on GenericDatabaseProvider).
• Preserves exact output array ordering while preventing a single 25-view dashboard batch request from seizing 25 connections from the SQL pool simultaneously.

---

Pillar 4: Multi-Replica Horizontal Scaling via MJ_SERVER_ROLE

4.1 Role Separation ('all' | 'api' | 'worker')

Configured via MJ_SERVER_ROLE environment variable or serverRole in mj.config.cjs:

MJ_SERVER_ROLE	HTTP / GraphQL / WebSocket Server	Background Polling Daemons (ScheduledJobs, TaskGraphDispatcher, IntegrationSyncWorker, WorkQueueHost, SessionJanitor, OrphanDetailReconciler)	Use Case
all (default)	Enabled	Enabled	Single-instance deployments & local development
api	Enabled	Disabled (stateless request/WS serving only)	Horizontally auto-scaled API pods behind a load balancer
worker	Healthcheck only (no public GraphQL/WS traffic)	Enabled	Dedicated background worker pods (1–2 replicas)

flowchart LR
    LB["Load Balancer (ALB / Envoy / NGINX)"]
    subgraph ApiTier["Stateless API Tier (MJ_SERVER_ROLE=api, 2–10 Replicas)"]
        API1["MJAPI Replica 1"]
        API2["MJAPI Replica 2"]
        API3["MJAPI Replica N"]
    end
    subgraph WorkerTier["Dedicated Worker Tier (MJ_SERVER_ROLE=worker, 1–2 Replicas)"]
        W1["MJAPI Worker 1<br/>• TaskGraphDispatcher<br/>• ScheduledJobsService<br/>• IntegrationSyncWorker<br/>• WorkQueueHost<br/>• SessionJanitor"]
    end
    Redis[("Redis Cluster<br/>Shared Cache & PubSub Bus")]
    SQLPrimary[("SQL Server / PostgreSQL<br/>Primary Read-Write")]
    SQLReplica[("SQL Read-Only Replica<br/>dbReadOnlyUsername")]

    LB --> API1
    LB --> API2
    LB --> API3
    API1 <--> Redis
    API2 <--> Redis
    API3 <--> Redis
    W1 <--> Redis
    API1 --> SQLPrimary
    API2 --> SQLPrimary
    API3 --> SQLPrimary
    API1 -.->|"Read-Only Queries"| SQLReplica
    API2 -.->|"Read-Only Queries"| SQLReplica
    API3 -.->|"Read-Only Queries"| SQLReplica
    W1 --> SQLPrimary

---

Configuration Reference (mj.config.cjs & Environment Variables)

mj.config.cjs

module.exports = {
  // Pillar 4: Server Role ('all' | 'api' | 'worker'), or set MJ_SERVER_ROLE env var
  serverRole: process.env.MJ_SERVER_ROLE || 'all',

  cacheSettings: {
    maxMemoryMB: 256,
    // Pillar 1: Pluggable Broadcast Provider
    // Built-in keys: 'SequencedRoleBucket' (default), 'AzureWebPubSub', 'AWSIoTCore', 'GCPPubSub'
    broadcastProvider: process.env.MJ_CACHE_BROADCAST_PROVIDER || 'SequencedRoleBucket',
    // Bounded RAM ring buffer size for Topology 1 sequence catch-up (default: 1000)
    invalidationRingBufferSize: 1000,
    // Optional overrides for smart entity broadcast filtering:
    alwaysBroadcastEntities: [],
    excludedBroadcastEntities: [],
    // Optional cloud multiplexer provider settings (when broadcastProvider is AzureWebPubSub / AWSIoTCore / GCPPubSub):
    broadcastProviderSettings: {
      // Azure Web PubSub:
      endpoint: process.env.AZURE_WEB_PUBSUB_ENDPOINT,
      accessKey: process.env.AZURE_WEB_PUBSUB_ACCESS_KEY,
      hubName: process.env.AZURE_WEB_PUBSUB_HUB || 'mj_cache_invalidation',
      // AWS IoT Core:
      iotEndpoint: process.env.AWS_IOT_DATA_ENDPOINT,
      region: process.env.AWS_REGION,
      accessKeyId: process.env.AWS_ACCESS_KEY_ID,
      secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
      // GCP Pub/Sub:
      projectId: process.env.GCP_PROJECT_ID,
      topicPrefix: process.env.GCP_PUBSUB_TOPIC_PREFIX || 'mj-cache-role',
      accessToken: process.env.GCP_ACCESS_TOKEN,
    },
  },
};

---

Apples-to-Apples Scalability Benchmarks (origin/next vs 4-Pillar Architecture)

Reproducible benchmark suite (packages/MJServer/src/__tests__/scalabilityBenchmarks.test.ts, executed via MJ_RUN_SCALABILITY_BENCHMARKS=1):

Pillar	Workload	Metric	origin/next Baseline	4-Pillar Architecture	Improvement
Pillar 1A	CacheInvalidation Fan-Out ($U=1,000$, 4 roles, 100 saves)	Permission + RLS evaluations	200,000 calls	640 calls	313x fewer evaluations
Pillar 1A	CacheInvalidation Fan-Out ($U=1,000$, 4 roles, 100 saves)	Fan-out CPU wall time	70.55 ms	19.23 ms	3.7x faster
Pillar 1A	CacheInvalidation Fan-Out ($U=5,000$, 4 roles, 100 saves)	Permission + RLS evaluations	1,000,000 calls	640 calls	1563x fewer evaluations
Pillar 1A	CacheInvalidation Fan-Out ($U=5,000$, 4 roles, 100 saves)	Fan-out CPU wall time	303.52 ms	107.41 ms	2.8x faster
Pillar 1B	PushStatusUpdates ($U=2,000$ sessions, 500 chunks)	Filter invocations & JSON.parse calls	1,000,000 filters / 1,000,500 JSON.parse	500 filters / 500 JSON.parse	2001x fewer JSON.parse calls
Pillar 1B	PushStatusUpdates ($U=2,000$ sessions, 500 chunks)	Routing + parse wall time	187.89 ms	0.88 ms	213.7x faster
Pillar 2A	1,000-row × 30-field GraphQL query (per query)	Middleware async wrapper calls	30,001 calls/query	1 call/query	30,001x fewer async wrappers
Pillar 2A	1,000-row × 30-field GraphQL query (10 runs)	Field resolution wall time	60.22 ms (6.02 ms/query)	6.22 ms (0.62 ms/query)	9.7x faster
Pillar 2B	5,000 per-request provider shells + queues	Array slot copies & RxJS Subjects	95,000 copies / 10,000 Subjects (5.10 ms)	0 copies (Object.create) / 0 Subjects (lazy) (2.21 ms)	2.3x faster
Pillar 2C	APIKeyEngine.ValidateAPIKey (500 warm requests)	DB RunViews & LastUsedAt Saves	1,000 RunViews + 500 Saves (578.8 ms)	2 RunViews + 1 Save (1.33 ms)	500x fewer DB reads & writes (435x faster)
Pillar 3	100 concurrent identical RunView calls (live SQL Server)	Physical SQL queries executed	100 SQL queries	1 SQL query	100x fewer SQL queries (99% DB load reduction)
Pillar 3	100 concurrent identical RunView calls (live SQL Server)	Total burst wall time / p50 / p95	1349.7 ms (p50: 764.1 ms, p95: 1288.4 ms)	329.5 ms (p50: 329.1 ms, p95: 329.3 ms)	4.1x faster burst completion