v6.1.0-edge.3: Materialized queries, a workflow debugger, and forms you can arrange

The fourth Edge build of the 6.1 line, and a large one — 104 changesets. Three things arrive that were previously impossible rather than merely awkward: a stored Query can be frozen into a physical table that is its own entity, a task graph can be stepped, paused and forced from inside Explorer, and a generated form’s chrome becomes something an installation arranges instead of something CodeGen dictates. Underneath, the task-graph engine took three rounds of adversarial hardening whose findings shared one signature — the run reported Complete while the wrong work happened.

Edge builds are prereleases. They publish under the edge dist-tag and never move latest.

New Features

• Query & Entity Materialization. Snapshot a stored Query’s result — or an entity’s base view — into a physical table that IS its own read-only entity, refreshed on a schedule behind an atomic wrapper view so readers never see a half-built table. Phase 2 adds parameterized read-time RowFilterBroad injection, so a caller can run a materialized parameterized Query with RunQueryParams.DataSource: 'Materialized' and still get row-level filtering.
• Workflow Run Console — a realtime runner and debugger for task graphs. A run draws what actually happened: queued, running and traveled edges, a left data panel, and a VCR toolbar. Breakpoints, edge overrides, force-complete and edit-input are wired into the console, so a held or interesting graph can be stepped without leaving Explorer. A Flow agent can be debugged from the Agent form’s Run dialog, starting paused at Submit.
• Embedded Records — an opt-in 1:1 owner-held companion on BaseEntity. A record and the peer its FK points at (Deal.OrderID → Order) load, validate and persist as one unit, with inverted save order so the owner’s FK is always satisfiable.
• Full-stack recursive foreign key support. CodeGen emits a four-function TVF suite per self-referencing FK on both SQL Server and PostgreSQL, projects Root<Field>/Depth/Path/IsLeaf/ChildCount into base views, and ships hierarchy traversal APIs. Opt-in via EntityField.Configuration → Hierarchy.IsHierarchy, so accidental self-references (LastRunID, ReplacedByID) don’t generate unwanted machinery.
• Form chrome is a five-layer stack you can arrange — CodeGen, app inclusions, the Auto ranker, an install overlay (MJ: Form Chrome Rules), and per-user rail order. A budgeted related-role ranker replaces all-in-More, “More” becomes a quiet overflow footer, and the left-nav rail collapses to a 36px spine, resizes, persists per entity, and marks the current section the way the rest of Explorer nav does. Form contributions can claim a related-entity grid or replace a named field panel.
• Metadata-driven pluggable authentication providers, replacing hardcoded provider wiring.
• Durable sync runs — lease/fence run ownership, DB-backed cancellation and progress, and an opt-in worker mode. “Is this run alive?” is now answerable from the database, and a stale sweep can make the original owner stop writing instead of racing it.
• @memberjunction/ng-graph-view — a reusable graph component with physics force relaxation, circular layouts, zoom/pan, search filtering and a cancelable Before/After event suite. @memberjunction/ng-hierarchy-tree lands alongside it, wired into 15 core entity form hierarchy panels.
• Entity.Configuration, EntityRelationship.Configuration and EntityField.Configuration — tenant-editable JSONType bags (IEntityConfiguration / IEntityRelationshipConfiguration / IEntityFieldConfiguration). NULL keeps existing behavior, so nothing changes until you set something.
• mj dev workspace and mj dev workspace doctor. The former generates the four parent files of a multi-repo Open App dev workspace; the latter is a read-only health check printing PASS/WARN/FAIL/SKIP per check — including a one-copy census that catches the duplicate-package failures this setup is prone to.
• An ephemeral staged binary upload pipeline, polymorphic related collections, and a file record viewer.
• Gantt gains zoom (named hour→year levels, Ctrl/Cmd+wheel, cancelable Before/After events), independent grid scrolling with a splitter and user-resized columns, double-click to open the underlying record, hover tooltips for truncated names, and DHTMLX theme variables mapped onto MJ semantic tokens so it follows light/dark and white-label automatically.
• Speech and search reach further — a second web-search provider for research agents ahead of the Google Custom Search shutdown, and find-agent actions now search in hybrid mode.
• A windowed chat transcript — load the latest page on open and prepend older pages from a top sentinel, with ConversationEngine’s full-history API unchanged for agents.

Improvements

• Task-graph engine hardening, rounds 1–3. Three adversarial review passes over the merged engine. Every finding shared one signature: the run reported Complete while the wrong work happened. Round 1 fixed four correctness defects at the seams between layers, Round 2 the ring of seams one layer out, Round 3 the incomplete fixes inside Round 2’s own repairs.
• Security hardening — SQL-filter validation, the OAuth callback handler, API-key lookup, and the new-user domain gate. Separately, SafeExpressionEvaluator is hardened against a sandbox escape.
• Three writes no longer stringify caller-built objects straight into a database row — an agent run’s Data and StartingPayload, and a task graph’s invocation envelope.
• PostgreSQL parity work. The deterministic integration tier can now be run against PostgreSQL, and the runtime and conversion defects that exposed are fixed. Identifier quoting is corrected for column names colliding with SQL keywords, and the two divergent tokenizers are consolidated into one. IS-A (Table-Per-Type) inheritance no longer silently does nothing on PostgreSQL — both defects were caught-and-logged, so CodeGen exited zero while every declared IS-A relationship was skipped.
• PostgreSQL counterparts for this release’s migrations, verified the only way that counts: a clean mj migrate of all 63 migrations against a database built from nothing, followed by mj sync push writing 13,849 records with zero errors.
• Connector runs get deadlines and honest sampling. A whole-run deadline means a hung stage fails instead of running forever; a hung discovery can no longer make a connector permanently unrefreshable; discovery honours the discoveryMaxRecords setting it already exposed; and a connector is told when it is being sampled so a sample cannot silently become an exhaustive walk.
• FetchChanges stops retrying its own timeout (which multiplied load on sources already too slow), and its per-page timeout is configurable rather than a hard-coded 30s.
• EntityField.Validate() validates value-list fields, so an IN (…) CHECK constraint finally has a runtime counterpart (#3969).
• GraphQL child-array FieldResolvers (Foo_BarIDArray) are no longer emitted. Load children via RunView or a hand-written mutation result type; CurrentUser now returns a first-class Roles field.
• Runtime Schema Update runs publish to the integration progress stream, so an RSU is observable the way a sync or a connector build already was.
• Related-entity grids behave like grids. They hug toolbar + header + rows instead of a 200px floor, prefill every join field on a new child record and persist those defaults on the URL so the link survives refresh and deep-linking, and can hide New/search/refresh chrome when a related list shouldn’t offer them.
• A read-mode form toolbar Refresh that reloads the current record in place and broadcasts to already-visible related grids, the IS-A side panel, and custom panels.
• The #3251 sequencing invariant is enforced on both suites, not just the deterministic one. It was a property of the process all along; the guard only read one suite, so a live-suite violation passed 196 green tests and surfaced only on a release run.

Bug Fixes

• An agent could die mid-run, and scheduled-job dispatch could stop entirely — __mj_CreatedAt?.getTime is not a function. Root cause was BaseEngine.OnExternalCacheChange poisoning entity_object caches by assigning serialized rows straight into engine properties, so date fields held ISO strings. Affects every engine with CacheLocal: true. A poisoned cache stopped all scheduled jobs on every poll until reload.
• A converted metadata-sync migration called PostgreSQL CRUD sprocs in the wrong argument shape, making a release fail to apply with function __mj.spUpdateEntity(...) does not exist. The converter chose the shape from the call’s argument count, but the shape belongs to the function; it now resolves it from pg_proc at apply time.
• T-SQL→PostgreSQL conversion defects that broke open-app installs.
• The queue engine never recorded a task’s terminal status — spUpdateQueueTask carried no EXECUTE grant, so the write failed, the row kept its prior status, and the in-memory task still reported Complete because the boolean Save() return was discarded.
• MJ: User Routine Runs rows are guaranteed to reach a terminal status, and the dispatcher can no longer report a status the database does not hold.
• Search results are verified to belong to the entity they are attributed to.
• $-sequence corruption in String.prototype.replace calls carrying runtime data (#3171).
• Graph node/edge record IDs compare with UUIDsEqual() rather than ===.
• IS-A hydration uses owned fields only — loading an Accounting Company Profile used to SetMany the whole child view row onto MJ: Companies and throw field-not-found; finalizeSave had the mirror-image defect.
• The Files custom form is no longer tree-shaken out of Bootstrap / BootstrapLite.
• New-record tabs no longer open twice. They store an empty recordId while the URL uses the new sentinel, and URL sync treated those as different records.
• Colocated vector stores are usable, and an index’s declared dimensions are no longer ignored.
• Box and Dropbox SDK errors log named diagnostic fields rather than the whole error object.
• Seeded rows like TagCloudViewType no longer dump a fallback warning at Explorer boot — ClassFactory.CreateInstance is skipped when a view-type DriverClass has no registered descriptor.