Member Junction
    Preparing search index...

    Process-wide watchdog that guarantees no AIAgentRun is left stuck in Status='Running' after the owning process dies (restart, crash, OOM) or its terminal-state write fails.

    Three cooperating mechanisms, all anchored to the database clock so they remain correct across multiple MJAPI instances behind a load balancer (no instance trusts its own clock):

    1. Heartbeat — while a run is in flight, the owning process stamps LastHeartbeatAt every AgentRunWatchdogConfig.heartbeatIntervalMs via SP_HEARTBEAT.
    2. SweepSweepOrphanedRuns force-fails any Running run whose heartbeat has gone stale, via SP_SWEEP (one atomic, set-based UPDATE … WHERE Status='Running' AND <stale> inside the proc — so concurrent sweeps from multiple instances are harmless and a run another instance is still heart-beating can't be caught mid-sweep). Run once on boot (closes restart-orphans) and on a timer (closes mid-life orphans).
    3. Graceful shutdown — on SIGTERM/SIGINT (via ShutdownRegistry) this process marks the runs it owns Cancelled via SP_CANCEL, closing the deploy case instantly.

    Every proc filters Status='Running' only — Paused and AwaitingFeedback are legitimately-not-progressing states and are never touched. The watchdog accesses the DB only through these stored procedures (writes) and the base view (reads) — never the base table — so it works as the regular runtime DB user, not just the elevated CodeGen user. Proc calls are built through the provider's Dialect.ProcedureCallSyntax, so they run on SQL Server and PostgreSQL unchanged.

    Hierarchy (View Summary)

    Implements

    Index

    Constructors

    Properties

    ShutdownName: "AgentRunWatchdog" = 'AgentRunWatchdog'

    Optional human-readable identifier surfaced in shutdown logs and used for de-duplication when Register is called repeatedly with the same instance but no name. Falls back to the constructor name.

    Accessors

    • get GlobalKey(): string

      Returns string

    Methods

    • The Global Object Store is a place to store global objects that need to be shared across the application. Depending on the execution environment, this could be the window object in a browser, or the global object in a node environment, or something else in other contexts. The key here is that in some cases static variables are not truly shared because it is possible that a given class might have copies of its code in multiple paths in a deployed application. This approach ensures that no matter how many code copies might exist, there is only one instance of the object in question by using the Global Object Store.

      Returns GlobalObjectStore

    • IShutdownable: on graceful shutdown, mark every run this process still owns as Cancelled (via SP_CANCEL) so it doesn't linger as a phantom Running row until the next sweep. Idempotent and never throws — a failed cancel just falls through to the staleness sweep later.

      Returns Promise<void>

    • Begin guarding a run. The owning agent calls this immediately after the run row is saved (so it has a stable ID). Idempotent per run ID. Capturing the provider/user lets the central heartbeat + shutdown statements run without each run re-supplying them.

      Parameters

      Returns void

    • Stop guarding a run. Optional — the periodic prune drops terminal runs automatically — but callers may invoke it for prompt cleanup right after writing a terminal state.

      Parameters

      • runID: string

      Returns void

    • Returns the singleton instance of the class. If the instance does not exist, it is created and stored in the Global Object Store. If className is provided it will be used as part of the key in the Global Object Store, otherwise the actual class name will be used. NOTE: the class name used by default is the lowest level of the object hierarchy, so if you have a class that extends another class, the lowest level class name will be used.

      Type Parameters

      Parameters

      • this: new () => T
      • OptionalclassName: string

      Returns T

    • Force-fail any Running agent run whose liveness heartbeat has gone stale (or that never beat and was started long ago), via the dedicated SP_SWEEP proc. The proc does the whole thing as one atomic, set-based UPDATE and returns the count, so it's safe to run on every boot and on a timer, on every instance: the predicate is staleness-based (never "fail all Running") and re-evaluated at write time, so it cannot touch a run a healthy instance is still heart-beating. Returns the number of rows failed.

      Parameters

      Returns Promise<number>