v6.1.3: Flow agents return results, atomic metadata push, dispatcher under least privilege

TL;DR

Flow agents can be sub-agents and return a result to their caller again, mj sync push is all-or-nothing again and no longer deadlocks against itself, and the durable task-graph dispatcher claims tasks under a least-privilege login; plus installer, metadata pull and multi-server cache fixes.

Upgrade Notes apply: one schema migration adds the dispatcher’s stored procedures, and mj sync push now holds one transaction for the whole push.

6.1.3 is the certification candidate for the 6.1 line. It replaces 6.1.2, which failed the upgrade gate on the first two fixes below.

MemberJunction 6.x is released under the Business Source License 1.1; 5.x and earlier stay ISC. What the licence grants, and what needs certification, is in the licence section of the v6 upgrade guide.

Bug Fixes

• Flow agents can be sub-agents and return their result again (@memberjunction/ai-agents). Since 6.1.0 a Flow agent compiled to a task graph and returned as soon as its steps were scheduled, so a caller that needed the answer got nothing: as a sub-agent it was refused outright, and as a root run it returned before any step had run. A run with a parentRun, or with agentTypeParams.executionMode: 'inRun', now walks its steps in-process and returns the final payload, including runs that start at a specific step. A top-level run with no option set is still dispatched to the task-graph dispatcher. Each run records which mode it used as a Decision step. A flow containing a Human step remains dispatch-only.
• mj sync push is all-or-nothing again (@memberjunction/metadata-sync). Since 6.1.0 only deletes rolled back on a failed push; creates and updates committed as each was saved, the push still reported a clean rollback, and the metadata files were restored from backup, so the files and the database no longer agreed. Every create, update and delete now runs in one transaction, one JSON-root graph at a time, and a failure anywhere rolls back everything the push wrote. Every record error stops the push, including records that fail without throwing. Incremental state is saved only after the commit. The previous per-graph parallel behaviour is available per entity with push.isolatedTransactions: true in that directory’s .mj-sync.json, or for one run with --isolated-transactions.
• A push no longer deadlocks against itself (@memberjunction/metadata-sync). With ten graphs running in parallel on separate connections, two concurrent creates of the same entity could each hold a lock on the row it inserted while the generated create procedure’s read-back through the entity view waited on the other’s, which SQL Server resolved by killing one push. Running the push on one connection removes the second session the cycle needed.
• Deferred work runs after the transaction commits, not after it ends (@memberjunction/core, @memberjunction/sqlserver-dataprovider, @memberjunction/actions). Durable After-save entity actions with no queue submitter polled the transaction depth and ran once it reached zero. A rollback reaches zero as surely as a commit, so they fired against rows that no longer existed, and under a long transaction they busy-spun for its whole length. Providers now carry a post-commit queue: a task registered inside a transaction waits for the outermost commit and is discarded on rollback, and a dispatcher that registers after its transaction has already settled passes a token captured at dispatch so the task follows the transaction that caused it. SQL Server’s deferred Entity AI Action queueing uses the same queue.
• The durable task-graph dispatcher claims tasks under a least-privilege login (@memberjunction/task-graph). Its guarded writes ran as raw SQL against the Task and AIAgentRun base tables, which the runtime roles are never granted, so on a database using the cdp_Developer or cdp_Integration login every claim was refused. The refusal came back as zero rows, the same result as losing a race to another instance, so the dispatcher concluded another instance owned every task and skipped them forever: dispatched workflows and scheduled graph work were inert, with nothing in the logs but a permission denial. The statements now live in stored procedures granted to those roles, with the predicates, rowcount arbitration and single-clock lease rule unchanged. A refused write is reported as a failure, the dispatcher stops the wave and says so, and it probes for the EXECUTE grant at startup. PostgreSQL was not affected.
• mj install no longer reports a working install as failed (@memberjunction/installer, @memberjunction/codegen-lib). CodeGen’s AFTER commands end with a 30-second npm start boot check of MJAPI. A server never exits on its own, so the timeout was the only way the check could end, and a timeout was recorded as failure, so every distribution install that reached CodeGen printed Installation failed on a system where the API had booted. Commands can now declare isDaemon, which makes staying up for the whole timeout the pass; exiting early is still a failure. The shipped distribution config and the built-in default declare it for the boot check.
• mj install reads turbo’s failure summary correctly (@memberjunction/installer). Both installer phases parsed turbo’s one Failed: line with a pattern that read only the first package and matched nothing at all when FORCE_COLOR added colour codes. A real MJAPI build failure listed behind a generated package was tolerated, and on colourised output the expected pre-CodeGen state of every install became a hard failure. One classifier now strips colour codes and reads the whole list, and installer-spawned turbo runs pin FORCE_COLOR=0. Also fixed: preflight probed for pnpm from a target directory it had not created yet and reported “pnpm not found”; five interactive prompts were pre-answered and could never fire, so DatabaseTrustCert wrote an empty value and migrate failed against self-signed SQL Servers; and the database phase reported connectivity it had not checked and a missing sa user on a correct setup.
• mj sync pull no longer loses records for an entity whose class is not registered (@memberjunction/metadata-sync). Pull read each record’s key through the entity’s typed property. When the generated subclass was not loaded in the CLI process, such as an Open App entity, every key read undefined, all records shared one key, and a pull of N new records wrote exactly one with an empty primaryKey. Keys are now read through the metadata-driven Get(), a record whose key genuinely has no value stops the pull, and a | inside a key value no longer collides with the match separator.
• Engine derived state survives a cross-server cache update (@memberjunction/core, @memberjunction/ai-engine-base). In a multi-server deployment with Redis pub/sub, applying a peer’s cache payload replaced an engine’s array with new entity objects but did not rebuild what the engine had derived from the old ones, so grouped child collections went empty while every row count stayed correct, and the process stayed that way until restart. For the AI engine this surfaced as “No suitable model found” on every request whenever another server warmed its cache. Derived state is rebuilt on both update paths, the property-change notification fires after the rebuild, and the AI engine’s rebuild is idempotent so repeated events no longer grow its collections.

Upgrade Notes

• One schema migration applies on mj migrate: it adds the spTaskGraph* stored procedures and grants EXECUTE on them to cdp_Developer and cdp_Integration. No table, view or column changes. It keeps its v6.2.x file name and is the same file that ships in 6.2, so a database upgraded to 6.1.3 skips it on a later move to 6.2. No PostgreSQL counterpart is needed; PostgreSQL installs already grant table access.
• mj sync push holds one transaction for the whole push. Writes in a shared directory are roughly 30 to 50 percent slower, a push with no changes is not, and locks are held until the push commits, so other sessions reading those tables wait when read committed snapshot is off. A deadlock with another session now fails the whole push rather than one graph. A directory that needs the old parallel, commit-as-you-go behaviour sets push.isolatedTransactions: true in its .mj-sync.json; --isolated-transactions and --no-isolated-transactions override every file for one run. Order isolated directories before shared ones in directoryOrder when they touch the same rows. The “commit the successful changes?” prompt is gone.
• Flow agents route by caller. A Flow agent with a parentRun runs in-process; a top-level run is dispatched as before. A parent that sets executionMode: 'dispatch' on a sub-agent flow is refused, because the parent would continue before the work had happened. Separately, a path whose destination step is not Active is no longer followed in either mode, so a flow whose only satisfied path leads to a disabled step finishes with success where 5.x failed with “No active steps found”.
• Durable After-save entity actions wait for the commit. An action registered inside a transaction now runs after the outermost commit, and a commit returns only once its queued tasks have run, so a slow action delays the committer.
• The installer’s boot check only helps installs of this tag or later. The isDaemon flag lives in the distribution config fetched from the release tag, so an install of 6.1.2 still reports the false failure.