v6.1.0-edge.1: Unified workflow DAG engine, external agent harnesses, composite graph saves

The second Edge build of the 6.1 line. Its centrepiece is the unified workflow DAG engine, landed across Phases 0–8 — three superseded subsystems retired, task-graph execution moved server-side, and design-time flows converged with runtime task graphs into a single model. Alongside it: external agent harnesses as a new agent type, BaseEntity composite graph saves, and a metadata-cache fix that had been quietly disabling auto-refresh in every deployment.

Edge builds are prereleases. They publish under the edge dist-tag and never move latest.

New Features

• Unified workflow DAG engine (Phases 0–8). Phase 0 retires the legacy Workflow, Report and Scheduled Action subsystems, freeing the Workflow name and removing a parallel, non-functioning orchestration model. Phase 1 makes the task substrate report what actually happened. Phase 2 moves task-graph execution server-side and makes it invocation-agnostic, with the control plane becoming Remote Operations. Phase 3 makes durable task graphs a first-class agent primitive. Phase 4 converges design-time flows and runtime task graphs into one graph model. Phases 5–8 add the graph component and properties panel, WorkflowSpec (one object binding what runs to when it runs), the trigger layer, and the observability decision left open across three reviews.
• External agent harnesses — a new MJ agent type. @memberjunction/ai-agent-harness runs external coding agents as MJ agents, with adapters for Claude Code CLI, Codex, Gemini CLI, OpenCode, Pi and a generic stdio/JSON transport. Sandboxing ships as a provider interface with Docker and local-directory implementations. Permission policy is a declared capability rather than prefix-matched command patterns, and turns record real inputs and outputs against the response contract.
• Entity companions and composite graph saves on BaseEntity. DeclareRelatedRecords() lets a parent and its children save atomically through a single Save(), with dirty tracking, ordering, orphan handling and client/server parity. The two transaction mechanisms that were previously blind to each other are replaced by one provider-arbitrated primitive.
• Databricks external data source. @memberjunction/external-data-source-databricks adds a read-only driver for Databricks SQL Warehouses over the official @databricks/sql SDK, always HTTPS, with the SDK as an optional peer dependency.
• API-key-scoped row filters. RowFilterID on APIKeyScope and APIApplicationScope — nullable FKs into the existing RowLevelSecurityFilter catalog. Purely additive (NULL is today’s behavior), and the FK makes a filter undeletable while a live key references it, so a key cannot be silently un-filtered.
• Layered base views. An entity can now have both a generated base view and a custom view layered over it.
• Entity Action workflow extensions. EntityAction becomes the general workflow-hook substrate, with a safer and more diagnosable execution log.
• Mobile records UX in Explorer. Below 768px the records region runs headerless and the tab strip is replaced by a record bar plus a bottom-sheet switcher listing every open record, docked ones included. Split layouts flatten to a single stack at render time with persistence suppressed, so desktop-made splits survive phone sessions untouched. Ships mj-bottom-sheet as a new generic primitive.
• Theme-aware registry components, including visualization ramps and spec-level StyleOverrides.
• @memberjunction/ng-task-graph-editor — a generic graph component whose subject is TaskGraphSpec.

Improvements

• Angular peer dependencies are declared as ranges (^21.1.3), not exact pins. A peer declaration is a compatibility claim, not an install instruction: the exact pins falsely claimed incompatibility with every other Angular 21.x build, produced 502 peer-resolution errors under strict pnpm workspaces, and structurally blocked Angular security patches behind a full republish of every ng-* package.
• pnpm cutover — declared the dependencies npm’s hoisting had been silently supplying. Under npm, a package could import a module it never declared and still resolve it. Each of these was a latent bug that happened to work; they are fixed independently of the package manager, with no published API change.
• One look and one keyboard contract across MJ’s tab strips.
• CodeGen smart-field-identification tightened. Prompt-template defaults now search OFF behind a narrow whitelist, with an anti-pattern list for filter and narrative fields and an FTS-only Contains predicate. A new search-guardrails module enforces the rules in code — narrative-field block, per-entity cap, predicate normalization, entity-level default-off — so flag drift cannot return regardless of what the model emits.
• Entity semantic search standardizes on Provider.SearchEntity.
• The SearchScope permission resolver is replaceable.
• Entities declare, per verb, which may be written by SQL that bypasses BaseEntity.
• Open App migrations run per-migration by default, and the mode is selectable.
• CodeGen introspection is faster — vwSQLColumnsAndEntityFields is narrowed to user objects.
• MJServer cleans dist before building, so a deleted source file cannot strand a compiled orphan.
• PostgreSQL parity for the full release. All eight new SQL Server migrations ship verified PostgreSQL counterparts, checked against a fresh PG 16 database and compared to SQL Server for table, FK, view and CRUD equivalence.

Bug Fixes

• The metadata cache was refreshing roughly every 50 hours instead of every 3 minutes. databaseSettings.metadataCacheRefreshInterval is milliseconds, but MJServer passed it undivided into a seconds argument, which was then multiplied by 1000 again — so the cache effectively never auto-refreshed. This is the likely root cause of “stale metadata until MJAPI restart”. Both halves are fixed together; fixing only the first would have made the user cache hammer the database every 180 ms.
• Action Filters were a no-op. RunSingleFilter was a return true stub, so every Action Filter passed. Filters now resolve through a registered BaseActionFilter subclass or by evaluating the filter’s Code, cached per row version, and fail closed: a filter that throws, returns a non-boolean, or has no evaluable logic prevents the action and logs why. No shipped metadata contains filter rows, so no existing behavior changes.
• Session-hijack in push-subscription delivery (B49) — statusUpdates is now bound to the authenticated connection identity.
• Hardened SQL text-building paths that substituted values into query strings without parameterization or escaping.
• A cost-guardrail fix affecting every agent type.
• AzureFileStorage now works for DB-credential (enterprise) deployments. The constructor required STORAGE_AZURE_* env vars and threw before initialize() could supply decrypted credentials, breaking the Archiving engine and the Explorer File Browser for any FileStorageAccount-backed Azure usage. Credentials are optional at construction; the env-var path is unchanged.
• The Explorer loading screen is always released, even when a resource’s load throws or hangs.
• UUID primary keys compare case-insensitively in BaseEngine cache maintenance — the SQL Server/PostgreSQL casing difference no longer causes misses.
• A failed Open App install now removes everything it wrote, not just its schema.
• Paginated sweeps are no longer falsely reported as duplicate RunViews, and telemetry exemption is honoured on batch reads.
• The Explorer session landing app is decoupled from the user-sortable Sequence order.
• @memberjunction/ng-notifications pin corrected from 5.51.0 to 6.0.0.
• Migration ordering corrected. AI_Agent_Harness_Foundation was retimestamped so it sorts after the previous Edge build’s applied ceiling. With Flyway’s outOfOrder defaulting to false, its original timestamp would have been silently skipped on every database upgrading from 6.1.0-edge.0 — fresh installs would have received the harness schema and upgraders would not.