Member Junction
    Preparing search index...

    Class VertexLLM

    VertexLLM - Google Vertex AI implementation

    Extends GeminiLLM and reuses all its logic (chat, streaming, thinking, parameters, multimodal content, message alternation, error handling, etc.).

    The only difference is authentication - this class overrides the constructor to handle GCP authentication instead of API keys.

    // Option 1: ADC (Application Default Credentials)
    // Set GOOGLE_APPLICATION_CREDENTIALS env var or use gcloud auth
    const llm = new VertexLLM(JSON.stringify({
      project: 'my-project',
      location: 'us-central1'
    }));

    // Option 2: Service account JSON as string (from credential system)
    const serviceAccountJson = JSON.stringify({
      type: 'service_account',
      project_id: 'my-project',
      private_key: '-----BEGIN PRIVATE KEY-----\n...\n-----END PRIVATE KEY-----\n',
      client_email: 'sa@my-project.iam.gserviceaccount.com',
      client_id: '123456789'
      // ... other service account fields
    });
    const llm = new VertexLLM(JSON.stringify({
      project: 'my-project',
      location: 'us-central1',
      serviceAccountJson: serviceAccountJson
    }));

    // Option 3: Service account JSON inline
    const llm = new VertexLLM(JSON.stringify({
      type: 'service_account',
      project_id: 'my-project',
      private_key: '-----BEGIN PRIVATE KEY-----\n...\n-----END PRIVATE KEY-----\n',
      client_email: 'sa@my-project.iam.gserviceaccount.com',
      client_id: '123456789',
      // ... other service account fields
      location: 'us-central1'
    }));

    // Option 4: Key file path reference. Honoured only when this JSON is the platform's environment key
    // (AI_VENDOR_API_KEY__VertexLLM); any other key that names a key file is refused when the client is created.
    const llm = new VertexLLM(JSON.stringify({
      keyFilePath: '/path/to/service-account.json',
      project: 'my-project',
      location: 'us-central1'
    }));

    Hierarchy (View Summary)

    Index

    Constructors

    constructor

    Properties

    _additionalSettings _gemini thinkingStreamState

    Accessors

    AdditionalSettings apiKey Credentials GeminiClient SupportsPrefill SupportsStreaming SupportsTools ThoughtSignatureEndpoint

    Methods

    addThinkingToMessage BuildCancelledChatResult ChatCompletion ChatCompletions ClassifyText ClearAdditionalSettings createClient createStreamingRequest extractThinkingFromContent finalizeStreamingResponse flushThinkingStreamRemainder geminiMessageSpacing GetFileCapabilities getThinkingTagFormat handleStreamingChatCompletion initializeThinkingStreamState IsCancellationError IsProviderAbortError isVolatileStateMessage nonStreamingChatCompletion processStreamChunkWithThinking processStreamingChunk resetStreamingState resetThinkingStreamState SetAdditionalSettings splitTrailingVolatileState SummarizeText supportsThinkingModels thoughtSignatureOrigin trailingVolatileStateIndex MapMJContentToGeminiParts MapMJMessageToGeminiHistoryEntry

    Constructors

    • Create a new VertexLLM instance

      Parameters

      • credentialsJson: string

        JSON string containing Vertex AI credentials. Must include at minimum: { project: 'id', location?: 'region' } Can include full service account JSON or keyFilePath for authentication (keyFilePath only in the environment key, AI_VENDOR_API_KEY__VertexLLM).

        If neither service account fields nor keyFilePath are provided, will use Application Default Credentials (ADC).

      Returns VertexLLM

    Properties

    Protected_additionalSettings

    _additionalSettings: Record<string, any>

    Protected property to store additional provider-specific settings

    Protected_gemini

    _gemini: GoogleGenAI
    thinkingStreamState: ThinkingStreamState

    State tracking for streaming thinking extraction Providers should initialize this if they support thinking models

    Accessors

    • get AdditionalSettings(): Record<string, any>

      Get the current additional settings

      Returns Record<string, any>

    • get apiKey(): string

      Only sub-classes can access the API key

      Returns string

    • get GeminiClient(): GoogleGenAI

      Read only getter method to get the Gemini client instance Note: This is async now because the client may not be initialized yet

      Returns GoogleGenAI

    • get SupportsPrefill(): boolean

      Whether this LLM provider supports assistant prefill (pre-seeding the start of the model's response). Providers that support prefill should override this to return true. This is used as a code-level default when database metadata (AIModelType/AIModel/AIModelVendor.SupportsPrefill) is null. Database values of true/false override this getter.

      Returns boolean

    • get SupportsStreaming(): boolean

      Gemini supports streaming

      Returns boolean

    • get SupportsTools(): boolean

      Gemini natively supports tool calling (functionDeclarations / functionCall / functionResponse parts).

      Returns boolean

    Methods

    • The result of a cancelled request, the same for every driver: a failed result with no choices, statusText 'cancelled', and an errorInfo that says Cancelled (Fatal, never failed over) — or, when the signal was aborted by a timeout, a retriable NetworkError.

      Parameters

      • startTime: Date
      • Optionaloptions: { error?: unknown; message?: string; signal?: AbortSignal }
        • Optionalerror?: unknown

          The error the abort raised, kept on exception.

        • Optionalmessage?: string

          The message to report; defaults to one naming the signal's reason.

        • Optionalsignal?: AbortSignal

          The request's abort signal, read for why it was aborted.

      Returns ChatResult

    • Process a chat completion request. If streaming is enabled and supported, this will route to the streaming implementation.

      Parameters

      Returns Promise<ChatResult>

    • Process multiple chat completion requests in parallel. This is useful for:

      • Generating multiple variations with different parameters (temperature, etc.)
      • Getting multiple responses to compare or select from
      • Improving reliability by sending the same request multiple times

      Parameters

      Returns Promise<ChatResult[]>

      Promise resolving to an array of ChatResults in the same order as the input params

    • Parameters

      • params: any

      Returns Promise<any>

      Classification through a driver method is deprecated along with the legacy AI Actions system, its only caller, and will be removed in the next major version. Most drivers do not implement it. Run an AI Prompt through AIPromptRunner (@memberjunction/ai-prompts) instead, which adds model selection, failover and cost tracking.

    • Clear all additional settings This is useful for resetting the state of the provider or when switching between different configurations.

      Returns void

    • Override parent's createClient() factory method to create Vertex AI client

      This factory method is called lazily on first use, so _credentials will always be set. The rest of the functionality (chat, streaming, etc.) remains identical.

      Returns Promise<GoogleGenAI>

    • Extract thinking content from non-streaming content This method handles case-insensitive extraction of thinking blocks

      Parameters

      • content: string

      Returns { content: string; thinking?: string }

    • Create the final response from streaming results for Gemini

      Parameters

      • accumulatedContent: string
      • lastChunk: any
      • usage: any

      Returns ChatResult

    • Returns (and clears) any user-visible content the thinking-tag stripper is still holding back at the end of a stream. Mid-stream, processStreamChunkWithThinking holds back a trailing fragment that could be the start of a <think>/</think> tag so a split tag never leaks as partial text; once the stream ends, such a fragment is real content and must be emitted (bug A5).

      Only flushes when NOT inside a thinking block: an unterminated <think> block's buffered text is reasoning, not answer, and is left held back (never surfaced as visible content). Returns '' when thinking extraction isn't active (no state) or there's nothing to flush.

      Returns string

    • Parameters

      • messages: Content[]

      Returns Content[]

    • Get the thinking tag format for this provider Providers can override this to customize the thinking tag format

      Returns { close: string; open: string }

    • Template method for handling streaming chat completion This implements the common pattern across providers while delegating provider-specific logic to abstract methods.

      Parameters

      Returns Promise<ChatResult>

    • Initialize thinking stream state for streaming extraction

      Returns void

    • Whether a request ended because it was aborted: its signal is aborted, the error is an abort by name (AbortError, APIUserAbortError, RequestAbortedError), or IsProviderAbortError recognizes it.

      Parameters

      • error: unknown
      • Optionalsignal: AbortSignal

      Returns boolean

    • Recognizes an abort error the shared names miss. A driver whose SDK raises its own class for an aborted request overrides this. Default: false.

      Parameters

      • _error: unknown

      Returns boolean

    • Whether a message is framework-authored volatile state for this request only: the loop agent's trailing runtime-state fragment (date/time, scratchpad, payload, a relocated specialization), which is rebuilt every iteration and must never sit inside a provider's cached prefix.

      The base test is the VolatileStateMessageMetadata.volatileState flag the agent layer sets. A driver may override to recognise the message by other means (Anthropic also accepts the fragment's tag literal for callers that pass plain text), but should call super first.

      Parameters

      Returns boolean

    • Process streaming chunk with thinking extraction This method handles case-insensitive extraction across chunk boundaries

      Parameters

      • rawContent: string

      Returns string

    • Process a streaming chunk from Gemini

      Parameters

      • chunk: any

      Returns { content: string; finishReason?: string; usage?: any }

    • Reset streaming state for a new request

      Returns void

    • Reset thinking stream state

      Returns void

    • Set additional provider-specific settings Subclasses should override this method to validate required settings

      Parameters

      • settings: Record<string, any>

        Provider-specific settings

      Returns void

    • Splits an outgoing request around its trailing volatile-state message, so a driver can place its cache boundary on the last message of head and send tail uncached. Returns null when the request has no trailing volatile state, in which case the driver formats the whole list as usual. Neither array is a copy of the input's messages; only the list is new.

      Parameters

      Returns TrailingVolatileStateSplit

    • Parameters

      Returns Promise<SummarizeResult>

      Summarization through a driver method is deprecated along with the legacy AI Actions system, its only caller, and will be removed in the next major version. Run a summarization AI Prompt through AIPromptRunner (@memberjunction/ai-prompts) instead, which adds model selection, failover and cost tracking.

    • Check if the provider supports thinking models Providers should override this to return true if they support thinking extraction

      Returns boolean

    • Endpoint plus model — the scope a thought signature validates in.

      Parameters

      • modelName: string

      Returns string

    • Index of the trailing volatile-state message in an outgoing request, or -1 when there is none. The fragment is the LAST message, or the one before it when an assistant prefill has been appended after it. -1 is also returned when nothing precedes the fragment, because then there is no stable history for a provider to cache ahead of it.

      Parameters

      Returns number

    • Maps one MJ message to a Gemini history entry.

      Parameters

      • message: ChatMessage

        The message to map

      • OptionalsignatureOrigin: string

        Where this request is going (endpoint + model). When given, a replayed call's thought signature from anywhere else is swapped for the placeholder.

      Returns Content

    Settings

    Member Visibility

    On This Page

    Constructors
    Properties
    Accessors
    Methods