@memberjunction/ai-vectordb

A provider-agnostic abstraction layer for vector databases in MemberJunction. This package defines the VectorDBBase abstract class and all associated types that concrete vector database implementations must fulfill. It contains no provider-specific logic -- only the contracts, data structures, and query interfaces shared across all implementations.

Architecture

graph TD
    subgraph Abstraction["@memberjunction/ai-vectordb"]
        VDBB["VectorDBBase (abstract)"]
        VR["VectorRecord"]
        QO["QueryOptions / HybridQueryOptions"]
        QR["QueryResponse / ScoredRecord"]
        IDX["IndexDescription / IndexList"]
        BR["BaseResponse"]
    end

    subgraph Implementations["Provider Implementations"]
        PC["PineconeDatabase"]
        CUSTOM["Custom Provider"]
    end

    subgraph Consumers["Consuming Packages"]
        SYNC["ai-vector-sync"]
        DUPE["ai-vector-dupe"]
        CORE["ai-vectors (VectorBase)"]
    end

    PC -->|extends| VDBB
    CUSTOM -->|extends| VDBB
    SYNC --> VDBB
    DUPE --> VDBB
    CORE --> VDBB

    style Abstraction fill:#2d6a9f,stroke:#1a4971,color:#fff
    style Implementations fill:#2d8659,stroke:#1a5c3a,color:#fff
    style Consumers fill:#7c5295,stroke:#563a6b,color:#fff
Copy

All vector database providers extend VectorDBBase and implement its abstract methods. Consumer code programs against the base class, making it straightforward to swap providers without changing application logic.

Installation

npm install @memberjunction/ai-vectordb
Copy

Quick Start

import { VectorDBBase, VectorRecord } from '@memberjunction/ai-vectordb';

// Use any concrete provider that extends VectorDBBase
const db: VectorDBBase = new PineconeDatabase(apiKey);

// Insert a vector record
const record: VectorRecord = {
    id: 'doc-001',
    values: [0.1, 0.2, 0.3 /* ... */],
    metadata: { entity: 'Products', recordId: '12345' }
};
await db.createRecord(record);

// Query for similar vectors
const result = await db.queryIndex({
    vector: [0.1, 0.2, 0.3 /* ... */],
    topK: 10,
    includeMetadata: true
});

if (result.success) {
    for (const match of result.data.matches) {
        console.log(`Match: ${match.id} (score: ${match.score})`);
    }
}
Copy

VectorDBBase Abstract Class

All vector database providers must extend this class. The constructor requires an API key, which is validated (must be non-empty) and stored for subclass access via a protected getter.

classDiagram
    class VectorDBBase {
        <>
        #apiKey : string
        +constructor(apiKey: string)
        +SupportsHybridSearch : boolean
        +listIndexes()* IndexList
        +getIndex(params)* BaseResponse
        +createIndex(params)* BaseResponse
        +deleteIndex(params)* BaseResponse
        +editIndex(params)* BaseResponse
        +queryIndex(params)* BaseResponse
        +HybridQuery(params) BaseResponse
        +createRecord(record)* BaseResponse
        +createRecords(records)* BaseResponse
        +getRecord(params)* BaseResponse
        +getRecords(params)* BaseResponse
        +updateRecord(record)* BaseResponse
        +updateRecords(records)* BaseResponse
        +deleteRecord(record)* BaseResponse
        +deleteRecords(records)* BaseResponse
    }

    class PineconeDatabase {
        +listIndexes() IndexList
        +queryIndex(params) BaseResponse
    }

    class CustomProvider {
        +SupportsHybridSearch : boolean
        +HybridQuery(params) BaseResponse
    }

    VectorDBBase <|-- PineconeDatabase
    VectorDBBase <|-- CustomProvider

    style VectorDBBase fill:#2d6a9f,stroke:#1a4971,color:#fff
    style PineconeDatabase fill:#2d8659,stroke:#1a5c3a,color:#fff
    style CustomProvider fill:#2d8659,stroke:#1a5c3a,color:#fff
Copy

All abstract methods use union return types (BaseResponse | Promise<BaseResponse>) so subclasses can implement them as either synchronous or asynchronous.

Abstract Methods Reference

Index Operations

Record Operations

Virtual Methods (Non-Abstract)

Hybrid Search

Hybrid search combines dense vector similarity (semantic search) with sparse keyword matching (BM25) to produce more relevant results than either technique alone. This is particularly useful when queries contain specific terms (product names, codes, identifiers) that benefit from exact text matching alongside semantic understanding.

How It Works

1. Check support: Read the SupportsHybridSearch getter before calling HybridQuery().
2. Call HybridQuery: Pass both a vector embedding and an optional keyword query string.
3. Fallback behavior: If the provider does not override HybridQuery(), the base class automatically falls back to a pure vector queryIndex() call, ignoring keyword-specific parameters (KeywordQuery, Alpha, FusionMethod). This means consumer code does not need to branch on provider capability -- it is always safe to call HybridQuery().

HybridQueryOptions

HybridQueryOptions extends QueryParamsBase with the following additional fields:

Plus the inherited QueryParamsBase fields: topK, includeValues, includeMetadata, filter.

Implementing Hybrid Search in a Provider

import { VectorDBBase, HybridQueryOptions, BaseResponse } from '@memberjunction/ai-vectordb';

export class WeaviateDB extends VectorDBBase {
    public override get SupportsHybridSearch(): boolean {
        return true;
    }

    public override async HybridQuery(params: HybridQueryOptions): Promise<BaseResponse> {
        const results = await this.client.hybridSearch({
            vector: params.vector,
            query: params.KeywordQuery,
            alpha: params.Alpha ?? 0.7,
            fusionType: params.FusionMethod ?? 'rrf',
            limit: params.topK,
            includeMetadata: params.includeMetadata
        });
        return { success: true, message: 'Hybrid query complete', data: results };
    }

    // ... remaining abstract method implementations
}
Copy

Consumer Usage

const db: VectorDBBase = getVectorProvider();

// Safe to call regardless of provider -- falls back to pure vector search
const results = await db.HybridQuery({
    vector: embedding,
    KeywordQuery: 'quarterly revenue report',
    Alpha: 0.6,
    topK: 20,
    includeMetadata: true
});

// Or check support for conditional UI behavior
if (db.SupportsHybridSearch) {
    showKeywordSearchInput();
}
Copy

Implementing a Custom Provider

Follow these steps to add support for a new vector database (e.g., Weaviate, Qdrant, Milvus).

Step 1: Create the Provider Class

import {
    VectorDBBase,
    BaseRequestParams,
    BaseResponse,
    CreateIndexParams,
    EditIndexParams,
    IndexList,
    QueryOptions,
    UpdateOptions,
    VectorRecord
} from '@memberjunction/ai-vectordb';

export class QdrantDB extends VectorDBBase {
    private client: QdrantClient;

    constructor(apiKey: string, host: string) {
        super(apiKey); // Validates the API key
        this.client = new QdrantClient({ apiKey: this.apiKey, host });
    }

    async listIndexes(): Promise<IndexList> {
        const collections = await this.client.getCollections();
        return {
            indexes: collections.map(c => ({
                name: c.name,
                dimension: c.vectors.size,
                metric: c.vectors.distance,
                host: this.host
            }))
        };
    }

    async getIndex(params: BaseRequestParams): Promise<BaseResponse> {
        const info = await this.client.getCollection(params.id);
        return { success: true, message: 'Index retrieved', data: info };
    }

    async createIndex(params: CreateIndexParams): Promise<BaseResponse> {
        await this.client.createCollection(params.id, {
            vectors: { size: params.dimension, distance: params.metric }
        });
        return { success: true, message: 'Index created', data: null };
    }

    async deleteIndex(params: BaseRequestParams): Promise<BaseResponse> {
        await this.client.deleteCollection(params.id);
        return { success: true, message: 'Index deleted', data: null };
    }

    async editIndex(params: EditIndexParams): Promise<BaseResponse> {
        await this.client.updateCollection(params.id, params.data);
        return { success: true, message: 'Index updated', data: null };
    }

    async queryIndex(params: QueryOptions): Promise<BaseResponse> {
        const results = await this.client.search(params);
        return { success: true, message: 'Query complete', data: results };
    }

    async createRecord(record: VectorRecord): Promise<BaseResponse> {
        await this.client.upsert([record]);
        return { success: true, message: 'Record created', data: null };
    }

    async createRecords(records: VectorRecord[]): Promise<BaseResponse> {
        await this.client.upsert(records);
        return { success: true, message: `${records.length} records created`, data: null };
    }

    async getRecord(params: BaseRequestParams): Promise<BaseResponse> {
        const record = await this.client.retrieve(params.id);
        return { success: true, message: 'Record retrieved', data: record };
    }

    async getRecords(params: BaseRequestParams): Promise<BaseResponse> {
        const records = await this.client.retrieveMany(params.data);
        return { success: true, message: 'Records retrieved', data: records };
    }

    async updateRecord(record: UpdateOptions): Promise<BaseResponse> {
        await this.client.update(record);
        return { success: true, message: 'Record updated', data: null };
    }

    async updateRecords(records: UpdateOptions): Promise<BaseResponse> {
        await this.client.updateMany(records);
        return { success: true, message: 'Records updated', data: null };
    }

    async deleteRecord(record: VectorRecord): Promise<BaseResponse> {
        await this.client.delete(record.id);
        return { success: true, message: 'Record deleted', data: null };
    }

    async deleteRecords(records: VectorRecord[]): Promise<BaseResponse> {
        await this.client.deleteMany(records.map(r => r.id));
        return { success: true, message: 'Records deleted', data: null };
    }
}
Copy

Step 2: Register with MemberJunction Class Factory (Optional)

If you want MemberJunction's class factory to discover your provider automatically:

import { RegisterClass } from '@memberjunction/global';

@RegisterClass(VectorDBBase, 'QdrantDB')
export class QdrantDB extends VectorDBBase {
    // ... implementation
}
Copy

Step 3: Add Hybrid Search Support (Optional)

If your vector database supports hybrid search natively, override the getter and method:

export class QdrantDB extends VectorDBBase {
    public override get SupportsHybridSearch(): boolean {
        return true;
    }

    public override async HybridQuery(params: HybridQueryOptions): Promise<BaseResponse> {
        // Use your provider's native hybrid search API
        const results = await this.client.hybridSearch({
            vector: params.vector,
            keyword: params.KeywordQuery,
            alpha: params.Alpha ?? 0.7,
            limit: params.topK
        });
        return { success: true, message: 'Hybrid query complete', data: results };
    }
}
Copy

QueryOptions Reference

QueryOptions is a union type supporting two query strategies:

graph LR
    QPB["QueryParamsBase
topK, includeValues,
includeMetadata, filter"]
    QBV["QueryByVectorValues
+ vector"]
    QBID["QueryByRecordId
+ id"]
    QO["QueryOptions"]

    QPB --> QBV
    QPB --> QBID
    QBV --> QO
    QBID --> QO

    style QPB fill:#2d6a9f,stroke:#1a4971,color:#fff
    style QBV fill:#2d8659,stroke:#1a5c3a,color:#fff
    style QBID fill:#2d8659,stroke:#1a5c3a,color:#fff
    style QO fill:#b8762f,stroke:#8a5722,color:#fff
Copy

QueryParamsBase

Shared query parameters inherited by both query strategies.

QueryByVectorValues

Query by providing a vector directly. Extends QueryParamsBase.

QueryByRecordId

Query using an existing record's stored vector. Extends QueryParamsBase.

Usage Examples

// Query by vector values
const resultByVector = await db.queryIndex({
    vector: [0.1, 0.2, 0.3],
    topK: 5,
    includeMetadata: true,
    filter: { category: 'finance' }
});

// Query by record ID (find similar to an existing record)
const resultById = await db.queryIndex({
    id: 'doc-001',
    topK: 10,
    includeMetadata: true
});
Copy

Type Reference

Core Types

Index Types

Request/Options Types

Query Types

Distance Metrics

The IndexModelMetricEnum supports three metrics for similarity comparison:

Available Implementations

Create additional implementations by extending VectorDBBase and optionally registering with MemberJunction's class factory.

Dependencies

Development

# Build
cd packages/AI/Vectors/Database
npm run build

# Run tests
npm run test

# Watch mode
npm run test:watch
Copy

License

ISC