AbstractProtected_The ID of the FileStorageAccount this driver instance is operating for. Set during initialization via the config parameter.
Protected_The name of the FileStorageAccount (for logging/display purposes).
Protected Abstract ReadonlyproviderThe name of this storage provider, used in error messages and logging. Each implementation must define this property with a descriptive name.
Gets the account ID this driver instance is operating for. Returns undefined if the driver was not initialized with an account.
Gets the account name this driver instance is operating for. Returns undefined if the driver was not initialized with an account.
AbstractIsChecks whether this storage provider is properly configured and ready to use.
This abstract getter must be implemented by each provider to verify that all required configuration parameters (API keys, credentials, endpoints, etc.) are present and valid. This allows the system to determine which providers are actually available before attempting to use them.
Implementation Guidelines:
Examples by Provider:
true if the provider is fully configured and ready to use, false otherwise
// In Google Drive provider:
public get IsConfigured(): boolean {
return !!(this._clientID && this._clientSecret && this._refreshToken);
}
// In AWS S3 provider:
public get IsConfigured(): boolean {
return !!(this._accessKeyId && this._secretAccessKey && this._bucketName);
}
// Usage:
const storage = new GoogleDriveFileStorage();
if (storage.IsConfigured) {
// Safe to use this provider
await storage.SearchFiles('query');
} else {
console.log('Provider not configured, skipping');
}
Indicates whether this provider supports streaming/ranged downloads via GetObjectStream.
The base default is false. Streaming-capable drivers override this getter to
return true. Callers MUST check this getter before calling GetObjectStream,
or be ready to catch StreamingNotSupportedError.
true if the provider can serve ranged, streamed reads; false otherwise.
AbstractCopyCopies an object within the storage system.
Unlike MoveObject which removes the source object, this creates a copy while leaving the original intact. This is useful for creating backups or versions of files.
The name of the object to copy.
The name to assign to the copied object.
A Promise that resolves to a boolean indicating success of the copy operation.
AbstractCreateCreates a directory in the storage system.
For storage systems that don't natively support directories (like AWS S3 or Google Cloud Storage), this may create a zero-byte object with a trailing delimiter to simulate a directory. For systems with native directory support, this will create an actual directory.
The path of the directory to create. Should typically end with a delimiter (typically "/").
A Promise that resolves to a boolean indicating success of the directory creation.
AbstractCreateGenerates a pre-authenticated URL for downloading files from a storage provider.
This method abstracts the process of generating download URLs across different storage providers, offering a unified interface for obtaining these URLs. When called with the name of the file or object to download, it creates a URL that includes necessary authentication tokens directly in the URL, allowing for secure access without requiring additional authentication at download time.
The name of the object or file for which you want to generate a download URL. This is the name as it is known to the storage provider, and it will be used to locate the file and generate the URL.
A Promise that resolves to a string, which is the pre-authenticated download URL for the specified object or file. This URL can be used immediately for downloading the file without further authentication.
const downloadUrl = await CreatePreAuthDownloadUrl("report.pdf");
console.log(downloadUrl); // Use this URL to download your file directly
If a ProviderKey was previously returned by CreatePreAuthUploadUrl, use that as the
objectName instead of the object's natural name:
const downloadUrl = await CreatePreAuthDownloadUrl(file.ProviderKey);
console.log(downloadUrl); // Use this URL to download your file directly
This method simplifies the process of securely sharing or accessing files stored in cloud storage by providing a direct, pre-authenticated link to the file. It's particularly useful in applications where files need to be accessed or shared without navigating through the storage provider's standard authentication flow each time.
AbstractCreateGenerates a pre-authenticated URL for uploading files to a storage provider.
This method abstracts over different storage providers, allowing for a unified interface to obtain upload URLs, regardless of the underlying provider's specifics. The method takes the name of the file (or object) you wish to upload as input and returns a Promise. This Promise, when resolved, provides a payload containing two key pieces of information:
UploadUrl: The URL to which the file should be uploaded. This URL is pre-authenticated,
meaning it includes any necessary authentication tokens or signatures. The URL's format
and the authentication method depend on the storage provider being used.
ProviderKey (optional): Some storage providers assign their own unique key or name to
the uploaded object instead of using the name provided by the user. If the provider you
are using does this, the ProviderKey will be included in the payload. This key can be
useful for future reference to the object within the storage provider's system.
The name of the object or file to be uploaded. This name is used by the storage provider and may also be included in the pre-authenticated URL.
A Promise that resolves to a payload containing the upload URL and, optionally, the provider's object key. This payload allows you to proceed with uploading your file to the storage provider.
const uploadPayload = await CreatePreAuthUploadUrl("photo.jpg");
console.log(uploadPayload.UploadUrl); // Use this URL to upload your file
if (uploadPayload.ProviderKey) {
// If this is returned, use it as the `objectName` for `CreatePreAuthDownloadUrl` or `DeleteObject`
console.log(uploadPayload.ProviderKey);
}
Note: This method is abstract and must be implemented by a subclass that specifies the logic for interacting with a specific storage provider.
AbstractDeleteDeletes a directory and optionally all of its contents recursively.
This method can be used to remove empty directories or, with the recursive flag set to true, to delete entire directory trees including all contained files and subdirectories.
The path of the directory to delete.
Optionalrecursive: booleanIf true, deletes all contents recursively. If false and the directory is not empty, the operation will fail. Defaults to false.
A Promise that resolves to a boolean indicating success of the deletion operation.
AbstractDeleteDeletes an object or file from a storage provider's system.
This method provides a unified interface for object deletion across different storage providers. It takes the name of the object to delete and handles the provider-specific operations to remove it from storage.
The name of the object or file to be deleted. This is the identifier used by the storage provider to locate the object for deletion.
A Promise that resolves to a boolean value. true indicates that the object was
successfully deleted, while false indicates a failure in the deletion process.
const deleted = await storage.DeleteObject('documents/old_report.pdf');
if (deleted) {
console.log('File successfully deleted');
} else {
console.log('Failed to delete file - it may not exist or there was an error');
}
If a ProviderKey was previously returned by CreatePreAuthUploadUrl, use that as the
objectName instead of the object's natural name:
const deleted = await storage.DeleteObject(file.ProviderKey);
if (deleted) {
console.log('File successfully deleted');
} else {
console.log('Failed to delete file');
}
AbstractDirectoryChecks if a directory exists in the storage system.
For storage systems that don't natively support directories, this may check for the existence of a directory placeholder object or for any objects with the given prefix.
The path of the directory to check.
A Promise that resolves to a boolean indicating if the directory exists.
AbstractGetDownloads an object's content as a Buffer.
This method retrieves the full content of an object from the storage provider and returns it as a Buffer for processing in memory.
Object identifier (prefer objectId for performance, fallback to fullPath)
A Promise that resolves to a Buffer containing the object's data. If the object does not exist, the promise will be rejected.
AbstractGetRetrieves metadata for a specific object without downloading its contents.
This method allows for checking properties of an object such as its size, content type, and last modified date without transferring the actual data, which can be efficient for large files.
Object identifier (prefer objectId for performance, fallback to fullPath)
A Promise that resolves to a StorageObjectMetadata object containing the metadata. If the object does not exist, the promise will be rejected.
try {
const metadata = await storage.GetObjectMetadata('documents/large_file.zip');
console.log(`File size: ${metadata.size} bytes`);
console.log(`Last modified: ${metadata.lastModified}`);
console.log(`Content type: ${metadata.contentType}`);
} catch (error) {
console.error('File not found or error retrieving metadata', error);
}
Streams an object's content (optionally a byte range) without buffering the whole object in memory.
Use this instead of GetObject when serving large media (audio/video) over
HTTP — pipe the returned ObjectStreamResult.Stream directly to the HTTP
response and propagate ContentType/ContentLength/ContentRange to the response
headers. GetObject returns the full object as a Buffer and is appropriate only for
small payloads you need fully in memory.
Range semantics: params.Range is an inclusive byte range
({ Start, End }), matching the HTTP Range/Content-Range model. Omit End to
stream from Start to EOF. When a range is honored, the result's ContentRange
carries the actual Start/End returned and the object's Total size — enough to
emit Content-Range: bytes Start-End/Total and a 206 Partial Content status. When
no Range is supplied, the whole object is streamed and ContentRange is omitted.
Capability gating: The base class provides a default implementation that throws
StreamingNotSupportedError. This means drivers that don't support streaming
inherit the throw automatically (no boilerplate stubs) and SupportsStreaming
stays false. Callers MUST therefore either check SupportsStreaming first, or be
prepared to catch StreamingNotSupportedError and fall back to GetObject.
Object identifier (prefer objectId, fall back to fullPath) plus an
optional inclusive Range.
A Promise resolving to an ObjectStreamResult with the readable stream and accompanying content metadata.
if (!storage.SupportsStreaming) {
// fall back to a full buffered read
}
const result = await storage.GetObjectStream({
objectId: 'media/big-video.mp4',
Range: { Start: 0, End: 1_048_575 }, // first 1 MiB, inclusive
});
res.status(result.ContentRange ? 206 : 200);
if (result.ContentType) res.setHeader('Content-Type', result.ContentType);
if (result.ContentLength != null) res.setHeader('Content-Length', String(result.ContentLength));
if (result.ContentRange) {
const { Start, End, Total } = result.ContentRange;
res.setHeader('Content-Range', `bytes ${Start}-${End}/${Total}`);
}
result.Stream.pipe(res);
Initialize storage provider with optional configuration.
ALWAYS call this method after creating a provider instance.
Constructor loads credentials from environment variables, then call initialize() with no config to complete setup:
// Set environment variables:
// STORAGE_AWS_ACCESS_KEY_ID=...
// STORAGE_AWS_SECRET_ACCESS_KEY=...
// STORAGE_AWS_BUCKET_NAME=...
// STORAGE_AWS_REGION=...
const storage = new AWSFileStorage(); // Constructor loads env vars
await storage.initialize(); // No config - uses env vars
await storage.ListObjects('/'); // Ready to use
Constructor loads defaults, then call initialize() with configuration to override with database credentials and track account:
// Preferred: Use infrastructure utility
const storage = await initializeDriverWithAccountCredentials({
accountEntity,
providerEntity,
contextUser
});
// Alternative: Manual initialization
const storage = new AWSFileStorage();
await storage.initialize({
accountId: account.id,
accountName: account.name,
accessKeyID: creds.accessKeyID,
secretAccessKey: creds.secretAccessKey,
region: creds.region,
defaultBucket: creds.bucket
});
Subclass implementations should:
Optionalconfig: StorageProviderConfigOptional configuration object - Omit for simple deployment (uses env vars from constructor) - Provide for multi-tenant (overrides with database credentials)
A Promise that resolves when initialization is complete
AbstractListLists objects in a storage provider's system with the given prefix path.
This method returns a list of objects (files) and prefixes (directories) under the specified path. It supports a hierarchical directory-like structure through the use of prefixes that can represent "folders" within the storage, even for providers that don't natively support directories.
The path prefix to list objects from. Use "/" for the root, or paths like "documents/" for specific directories.
Optionaldelimiter: stringThe character used to group keys. Typically "/" is used to simulate directory structure. Defaults to "/".
A Promise that resolves to a StorageListResult containing both objects and prefixes (directories).
// List all objects in the "documents" directory
const result = await storage.ListObjects('documents/');
// Access the files
for (const file of result.objects) {
console.log(`File: ${file.name}, Size: ${file.size}, Type: ${file.contentType}`);
}
// Access the subdirectories
for (const dir of result.prefixes) {
console.log(`Directory: ${dir}`);
}
AbstractMoveMoves an object or file from one location to another within a storage provider's system.
This method provides a unified interface for moving objects across different storage providers. It takes the original object name and the new desired name/location, and handles the appropriate operations to move the object while preserving its content.
Some implementations may perform this as a copy followed by a delete operation, while others might use native move capabilities of the underlying storage system.
The current name/path of the object to be moved. This is the identifier used by the storage provider to locate the object for moving.
The new name/path where you want to move the object. This should match exactly as it is expected to be known to the storage provider after moving.
A Promise that resolves to a boolean value. true indicates that the object was
successfully moved, while false indicates a failure in the move process.
AbstractObjectChecks if an object exists in the storage system.
This method provides a way to verify the existence of an object without transferring its data or metadata, which can be more efficient when you only need to know if something exists.
The name of the object to check.
A Promise that resolves to a boolean indicating if the object exists.
AbstractPutUploads object data to the storage provider.
This is a direct upload method that doesn't use a pre-authorized URL. Instead, it takes the data as a Buffer and handles the upload directly, making it suitable for server-side operations where you already have the data in memory.
The name to assign to the uploaded object.
The Buffer containing the data to upload.
OptionalcontentType: stringOptional MIME type of the content. If not provided, it may be inferred from the object name or set to a default.
Optionalmetadata: Record<string, string>Optional custom metadata to associate with the object.
A Promise that resolves to a boolean indicating success of the upload.
// Upload a text file
const content = Buffer.from('Hello, World!', 'utf8');
const uploaded = await storage.PutObject(
'documents/hello.txt',
content,
'text/plain',
{ author: 'John Doe', department: 'Engineering' }
);
if (uploaded) {
console.log('File uploaded successfully');
} else {
console.log('Failed to upload file');
}
AbstractSearchSearches for files across the storage system using the provider's native search capabilities.
This method leverages each provider's built-in search APIs to find files matching the specified query. The search can include filenames, file content, and metadata depending on the provider's capabilities and the options specified.
Provider Support:
Query Syntax: The query parameter accepts natural language search terms. Advanced query syntax (like boolean operators, exact phrases, wildcards) varies by provider:
Implementations should document their specific query syntax support.
The search query string. Can be a simple term, phrase, or provider-specific advanced query syntax.
Optionaloptions: FileSearchOptionsOptional search configuration parameters.
A Promise that resolves to a FileSearchResultSet containing matching files.
// Simple filename search
const results = await storage.SearchFiles('quarterly report');
for (const file of results.results) {
console.log(`Found: ${file.path} (${file.size} bytes)`);
}
// Search with filters
const pdfResults = await storage.SearchFiles('budget', {
fileTypes: ['pdf'],
modifiedAfter: new Date('2024-01-01'),
pathPrefix: 'documents/finance/',
maxResults: 50
});
// Content search (if supported)
const contentResults = await storage.SearchFiles('machine learning algorithm', {
searchContent: true,
fileTypes: ['docx', 'pdf', 'txt']
});
// Check if there are more results
if (contentResults.hasMore) {
console.log(`Found ${contentResults.totalMatches} total matches`);
console.log(`Use nextPageToken to fetch more results`);
}
ProtectedthrowHelper method to throw an UnsupportedOperationError with appropriate context. This method simplifies implementation of methods not supported by specific providers.
The name of the method that is not supported
Represents an abstract base class for file storage operations.
This class defines a common interface for interacting with various cloud storage providers, allowing for consistent file operations regardless of the underlying storage system. Each storage provider implementation extends this base class and provides concrete implementations for all abstract methods.
The class provides methods for:
Implementation notes: