Server-side entity subclasses for MemberJunction that provide extended functionality and business logic for core entities when running in a Node.js environment.
This package contains server-side implementations of MemberJunction entity subclasses that require server-specific functionality such as direct database access, file system operations, server-side API integrations, complex business logic, and automatic creation/management of related entities.
graph TD
A["Core Entities Server"] --> B["AIPromptEntityExtended"]
A --> C["MJQueryEntityServer"]
A --> D["ComponentEntityExtended"]
A --> E["MJActionEntityExtended"]
A --> F["ApplicationEntityExtended"]
A --> G["Other Server Entities"]
B -->|manages| H["Templates
Template Contents"]
C -->|extracts| L["Query Fields
Parameters
Entities
Dependencies"]
D -->|generates| I["Vector Embeddings"]
J["MJ Class Factory"] -->|returns server subclass| A
K["Core Entities
(base classes)"] -->|extended by| A
style A fill:#2d6a9f,stroke:#1a4971,color:#fff
style B fill:#7c5295,stroke:#563a6b,color:#fff
style C fill:#7c5295,stroke:#563a6b,color:#fff
style D fill:#7c5295,stroke:#563a6b,color:#fff
style E fill:#7c5295,stroke:#563a6b,color:#fff
style F fill:#7c5295,stroke:#563a6b,color:#fff
style H fill:#2d8659,stroke:#1a5c3a,color:#fff
style I fill:#b8762f,stroke:#8a5722,color:#fff
style J fill:#2d6a9f,stroke:#1a4971,color:#fff
style K fill:#b8762f,stroke:#8a5722,color:#fff
style L fill:#2d8659,stroke:#1a5c3a,color:#fff
npm install @memberjunction/core-entities-server
Each server-side subclass registers itself with @RegisterClass when the package is imported, so there is no setup function to call:
import '@memberjunction/core-entities-server';
Important: The package must be loaded before any entities are instantiated. An MJAPI server built on @memberjunction/server-bootstrap loads it through the bootstrap's class-registration manifest, which also keeps a bundler from removing the classes as unused. Other server applications import the package at startup.
The package includes an extended AI Prompt entity that automatically manages Template and Template Contents records:
import { Metadata } from '@memberjunction/core';
// The metadata system will automatically use the server-side subclass
const md = new Metadata();
const aiPrompt = await md.GetEntityObject<AIPromptEntity>('AI Prompts');
// Setting TemplateText will automatically create/update Template records
aiPrompt.TemplateText = `You are a helpful assistant...`;
// Save will handle all the Template management automatically
await aiPrompt.Save();
Extends the base AI Prompt entity with automatic Template management:
TemplateText - A convenient way to get/set the prompt template contentExtends the Component entity with automatic vector embedding generation:
FunctionalRequirementsVectorBinary / TechnicalDesignVectorBinary (float32 bytes), which readers decode much faster than the JSON vectorsThe server-side MJ: Queries entity. It extends MJQueryEntityExtended from @memberjunction/core-entities:
Parameters, Fields and Entities rows it declares save together as one graph.SkipEmbeddings is set.QueryExtractionService re-runs extraction for saved queries without changing their SQL. See Query Refresh.Extends the agent note entity to keep the in-process vector store in sync with persisted state:
Save() and Delete() so that AIEngine.Instance._noteVectorService contains an entry for a note iff its persisted Status='Active' and it has a stored vector (EmbeddingVectorBinary or EmbeddingVector)Active and any other value, or when a note is deleted, the corresponding vector store entry is added/removed in the same operation — eliminating the need for an MJAPI restart after note revocation (e.g., during MemoryManagerAgent consolidation or contradiction resolution)AIAgentNote columns introduced by V202604260056__v5.30.x__Memory_Consolidation_Schema.sql (ConsolidatedIntoNoteID, ConsolidationCount, DerivedFromNoteIDs, ProtectionTier, ImportanceScore) persist via the standard generated setters — no custom logic needed in this overrideSee @memberjunction/aiengine README for the read-side and the consuming code in @memberjunction/ai-agents.
Companion to MJAIAgentNoteEntityServer — maintains the parallel invariant for the agent example vector store on Save/Delete.
The AI-from-Description half of the Remote Operations primitive (RO-4). For a MJ: Remote Operations row with GenerationType='AI', the Save hook calls the Generate Remote Operation Code prompt to author the body of the operation's InternalExecute — against the ambient input / provider / user / context contract — stores it in Code, records the libraries it used in the JSONType LibrariesObject, and resets CodeApprovalStatus='Pending' for human review (skipped when CodeLocked). CodeGen then emits a complete registered class from the approved code. Mirrors MJActionEntityServer; transactional; no junction CRUD (the library list is a JSONType field).
The package uses MemberJunction's entity subclassing system:
@RegisterClass decorator registers it with the metadata systemServer-side entities can expose virtual properties that don't exist in the database but provide convenient access to complex data:
export class AIPromptEntityExtendedServer extends AIPromptEntity {
private _TemplateText?: string;
get TemplateText(): string | undefined {
return this._TemplateText;
}
set TemplateText(value: string | undefined) {
this._TemplateText = value;
}
}
src/custom/Example:
import { BaseEntity, RegisterClass } from '@memberjunction/core';
import { UserEntity } from '@memberjunction/core-entities';
@RegisterClass(BaseEntity, 'Users')
export class UserEntityExtendedServer extends UserEntity {
async BeforeSave(): Promise<boolean> {
// Server-side validation or processing
if (this.Email) {
// Verify email domain against company whitelist
const validDomain = await this.checkEmailDomain(this.Email);
if (!validDomain) {
throw new Error('Invalid email domain');
}
}
return super.BeforeSave();
}
private async checkEmailDomain(email: string): Promise<boolean> {
// Server-side logic to validate email domain
return true;
}
}
This package works seamlessly with:
@memberjunction/core: Provides the base entity system@memberjunction/core-entities: Provides the base entity classes@memberjunction/server: Can be used in MJ server applications@memberjunction/cli: Used by CLI tools like MetadataSyncThis package provides a utility helper for server-side entities to easily implement vector embedding generation using the MemberJunction AI Engine.
A shared utility function that simplifies embedding generation for any server-side entity:
import { EmbedTextLocalHelper } from '@memberjunction/core-entities-server';
import { BaseEntity, SimpleEmbeddingResult } from '@memberjunction/core';
@RegisterClass(BaseEntity, 'MyEntity')
export class MyEntityServer extends MyEntity {
protected async EmbedTextLocal(textToEmbed: string): Promise<SimpleEmbeddingResult> {
return EmbedTextLocalHelper(this, textToEmbed);
}
}
The helper:
SimpleEmbeddingResultVARBINARY(MAX) binary companion (see the Binary Fields Guide)public async Save(): Promise<boolean> {
await this.GenerateEmbeddingsByFieldName([
{
fieldName: "Content",
vectorFieldName: "ContentVector",
modelFieldName: "ContentVectorModelID",
binaryVectorFieldName: "ContentVectorBinary" // optional; set and cleared with the JSON field
}
]);
return await super.Save();
}
The note, example, component, query and tag entity servers all write both the JSON and the binary vector column. Readers should load with ReadStoredVector(binary, json) from @memberjunction/ai-vectors-memory, which prefers the binary column and falls back to JSON.
// AI Prompts automatically manage their Template records
const prompt = await md.GetEntityObject<AIPromptEntity>('AI Prompts');
prompt.TemplateText = "New prompt content";
await prompt.Save(); // Template and Template Contents created/updated automatically
// Components automatically generate embeddings for text fields
const component = await md.GetEntityObject<ComponentEntity>('MJ: Components');
component.FunctionalRequirements = "The system shall...";
component.TechnicalDesign = "Architecture overview...";
await component.Save(); // Embeddings generated automatically
@RegisterClass(BaseEntity, 'Documents')
export class DocumentEntityServer extends DocumentEntity {
async BeforeSave(): Promise<boolean> {
// Scan document for viruses using server-side scanner
if (this.Content) {
const isSafe = await virusScanner.scan(this.Content);
if (!isSafe) {
throw new Error('Document failed security scan');
}
}
return super.BeforeSave();
}
}
@RegisterClass(BaseEntity, 'Orders')
export class OrderEntityServer extends OrderEntity {
async AfterSave(): Promise<boolean> {
// Send order to fulfillment system
await fulfillmentAPI.submitOrder(this);
// Update inventory
await inventorySystem.decrementStock(this.OrderItems);
return super.AfterSave();
}
}
When a query is saved and it is new, or its SQL changed, MJQueryEntityServer works out what the query reads and returns, and writes it to the query's child entities. Nothing has to be filled in by hand for a query to be usable.
{{query:"Category/Name(...)"}}), and the tables and views it reads. Whether the query uses a template at all (UsesTemplate) comes from the lexer, so a tag inside a comment does not count.ParameterHints, keyed by parameter name), once for the base render and once for each {% if %} branch, and the database server is asked to describe each render without running it. The server reports each output column's exact type, nullability, and the table column it comes from, when there is one.LatestResult).| Child entity | What is written |
|---|---|
MJ: Query Fields |
Each output column: name, order, SQL type, nullability, whether it is computed, and its source (SourceEntityFieldID when the column maps to an entity field, otherwise SourceObjectName, the object the server reported) |
MJ: Query Parameters |
Each template parameter: name, inferred type, whether it is required, its default |
MJ: Query Entities |
Each entity whose base view, base table or alternate base view the query reads |
MJ: Query Dependencies |
Each query it composes, with the alias and parameter mapping |
SkipEntityAIActions: true in the save options to skip this step.Extraction separates what the SQL determines from what a person wrote:
DetectionMethod Manual), its type, required flag, default, sample value and validation filters. Doc comments in the SQL ({# @param ... #}, {# @field ... #}, and a trailing --: comment on a select-list line) count as authored.ExtractionMessage.Each child row records who wrote it:
| Value | Used on | Meaning |
|---|---|---|
Database |
Query Fields | The facts came from the database server describing the query's output |
Parser |
Query Fields, Parameters, Entities | The facts came from MJ's analysis of the template and SQL structure |
Manual |
Query Fields, Parameters, Entities, Dependencies | A person wrote the row. A Manual parameter's annotations are never overwritten, and it is kept when the SQL stops using it |
AI |
Query Fields, Parameters, Entities | On a field, an LLM wrote its description (an LLM never sets types or sources). On a parameter or entity link, the row was written by the earlier LLM-based extraction |
Auto |
Query Dependencies | The dependency was found in the SQL |
Each run is recorded on the MJ: Queries row:
| Column | Meaning |
|---|---|
ExtractionStatus |
OK: every fact was extracted. Warning: extraction succeeded with findings. Error: the database server could not describe the query, so the stored child rows were kept. Skipped: the facts could not be checked, for example for a query on an external data source. NULL: never extracted. |
ExtractionMessage |
The run's findings, one per line: flagged parameters, rule warnings, the server's error |
ExtractedSQLHash |
A hash of the SQL and dialect the child rows were extracted from. A value that differs from the current SQL's hash means the rows may be stale |
MJQueryEntityServer.ExtractionOverride sets what the next save does. It goes back to Auto after every save.
| Value | Behaviour |
|---|---|
Auto (default) |
Extract when the query is new or its SQL changed |
Skip |
Do not extract, even when the SQL changed. The save-time rules are not applied, and the child rows, the extraction columns and UsesTemplate stay as they are. For bulk loaders and fixtures that write those rows themselves |
Force |
Extract even when the SQL did not change. On a query whose SQL did not change, rule errors are recorded instead of refusing the save; a save that changes the SQL is still held to the save-time rules |
import { Metadata, UserInfo } from '@memberjunction/core';
import { MJQueryEntityServer } from '@memberjunction/core-entities-server';
async function saveQuery(contextUser: UserInfo, queryID: string, sql: string): Promise<boolean> {
const md = new Metadata();
const query = await md.GetEntityObject<MJQueryEntityServer>('MJ: Queries', contextUser);
if (!(await query.Load(queryID))) return false;
query.SQL = sql;
query.ExtractionOverride = 'Auto'; // the default; 'Skip' or 'Force' change this one save
const saved = await query.Save({ SkipEntityAIActions: true }); // no LLM description pass
if (!saved) {
console.error(query.LatestResult?.CompleteMessage); // the save-time rule findings
}
return saved;
}
Parameters, Fields and Entities collections, and the query and those rows save as one graph. Extraction does not add a second row with a declared row's name or delete a declared row. A declared field still gets its facts from the describe, so only its annotations are the caller's.Error rather than failing a save whose row is already written.ExtractionMessage says why.RunExtractionPipeline in src/custom/query-extraction/pipeline.ts, the earlier LLM-assisted extraction, is deprecated. It is not used by saves and is not exported from the package root. Saves use ComputeQueryExtraction with WriteExtractedChildRows. The pipeline is kept until 7.0 for code that imports it directly.
QueryExtractionService re-runs extraction for saved queries on demand, without changing their SQL. It is the same extraction a save runs, under the refresh rules: a rule error is recorded on the query, never a refusal. Use it after a schema change, after an upgrade, or when a query's child rows look stale.
RefreshQuery(queryID, options) refreshes one query and returns a QueryRefreshReport.RefreshQueries(filter, options) refreshes a selection and returns a QueryRefreshRunReport. The QueryRefreshFilter selectors combine as a union: QueryIDs, QueryNames (a bare name or a full path such as Category/Sub/Name), CategoryPath (that category and every category below it), EntityNames or EntityIDs (the queries that read those entities), or All. A filter with no selector refreshes nothing.QueryRefreshOptions: DryRun works out and reports the changes without writing anything; WithAI queues the LLM description pass for rows still without a description (off by default); OnProgress is called after each query in a bulk run.Each query is refreshed in its own transaction on the caller's provider, so a failure rolls back that query only. A bulk run refreshes the queries a composition depends on before the queries that compose them; queries that compose each other in a cycle are refreshed once, after the rest, and listed in Cycles.
Each query's report gives:
Outcome: Refreshed (rows changed, or would change in a dry run), Unchanged, NotPermitted (a bulk run skipped it), Unavailable (does not exist, or the caller may not refresh it), or Failed (nothing was written)Added, Updated and Removed: the child rows changed, with the changed columns for an updateFlagged: authored parameters the SQL no longer usesExtractionStatus, QueryColumns (the query-row columns that change) and Diagnosticsimport { IMetadataProvider, UserInfo } from '@memberjunction/core';
import {
FormatQueryRefreshRun,
IsQueryRefreshProvider,
QueryExtractionService,
QueryRefreshFilter,
} from '@memberjunction/core-entities-server';
async function refreshSalesQueries(provider: IMetadataProvider, contextUser: UserInfo): Promise<void> {
if (!IsQueryRefreshProvider(provider)) {
throw new Error('A query refresh needs a provider that can run views');
}
const service = new QueryExtractionService({ Provider: provider, User: contextUser });
const filter: QueryRefreshFilter = { CategoryPath: 'Reports/Sales' };
// Review first: a dry run reports what would change and writes nothing
const preview = await service.RefreshQueries(filter, {
DryRun: true,
OnProgress: p => console.log(`${p.Processed}/${p.Total} ${p.Report.Outcome}: ${p.Report.QueryName}`),
});
for (const line of FormatQueryRefreshRun(preview, true)) console.log(line);
// Then apply
const run = await service.RefreshQueries(filter);
for (const report of run.Reports) {
if (report.Outcome === 'Failed') console.error(report.QueryName, report.Diagnostics);
}
}
Every entry point goes through QueryExtractionService, so the same authorization applies to all of them:
Query.RefreshExtraction remote operation (QueryRefreshExtractionServerOperation): a long-running operation that reports each query as a progress event. An API key needs the query:refresh scope.mj query refresh [--id|--name|--category|--entity|--all] [--dry-run] [--with-ai]. After an upgrade, run mj query refresh --all --dry-run to review, then mj query refresh --all.RebindQueriesForChangedEntities refreshes the queries that read an entity whose fields were added, dropped, or changed in type or nullability, with no description pass. The SQL is never edited: a query the change broke is recorded as an Error with the server's message. A query authorized by its role list (MJ: Query Permissions) that gains a column from a field with field permissions is recorded as a Warning.MJ: Queries and Create, Update and Delete on MJ: Query Fields, MJ: Query Parameters, MJ: Query Entities and MJ: Query Dependencies. A dry run needs the same, since its report shows the same metadata. Run permissions on the query play no part: refreshing is an authoring act.Unavailable). A bulk run reports a refused query as NotPermitted, with no reason.UserCanRefreshQueries(user, entities) and UserCanSeeQuerySources(user, entityIDs, entities) expose these checks for callers that want to test them up front.
This package does not contain a SQL parser. SQL and template parsing live in @memberjunction/sql-parser, which query extraction uses. New code reads SQL with SqlDocument.Parse(sql, dialect) (table references, column references, output items, CTEs) and templates with QueryTemplate.Parse(sql, dialect) (parameters, compositions, {% if %} branches).
The legacy SQLParser class in @memberjunction/sql-parser is deprecated but kept. Its Parse and extraction methods now run on the new parser, which changes some results from earlier releases:
{{query:"..."}} composition is not a table.dbo on SQL Server, public on PostgreSQL). The dialect defaults to SQL Server when none is given.ParseWithTemplatePreprocessing is the same as Parse. UsedASTParsing is true when every reading of the SQL parsed completely, and Columns is empty when it could not be read completely.Ensure the package is imported before any entity is instantiated. In a bundled server, check that it is in the class-registration manifest (mj codegen manifest), so the bundler does not drop the classes.
This package is a native ES module ("type": "module"). Import it from ES module code (import), not with require().
Virtual properties are not saved to the database. Use them as convenient accessors that manage real database fields behind the scenes.
When contributing new server-side entities:
This package is part of the MemberJunction project.