Documentation for Vantage Integration: Index

Overview

The "index" integration in Vantage provides a unified interface for interacting with various database backends, including Firestore, MongoDB, DynamoDB, PostgreSQL, and more. It dynamically accommodates multiple database types allowing for flexible configuration and extensibility while enabling powerful data analytics and retrieval functionalities within the Vantage platform.

Settings

1. Database Adapter

• Name: databaseAdapter
• Input Type: Dropdown
• Description: This setting allows the selection of the database adapter that the integration will utilize for data interactions. The available options are firestore, mongo, dynamo, and pg.
• Default Value: firestore
• Effects of Change: Changing this setting modifies the underlying data source used for queries and operations. It impacts data retrieval methods and may require adapting query formats according to the chosen database's syntax.

2. Query Timeout

• Name: queryTimeout
• Input Type: Numeric (milliseconds)
• Description: This setting specifies the maximum time in milliseconds that a query should take before it is aborted.
• Default Value: 30000 (30 seconds)
• Effects of Change: Setting a lower timeout can prevent long-running queries from consuming resources unnecessarily but may lead to incomplete data retrieval if the queries aren’t optimized or the database is under heavy load.

3. Connection Pool Size

• Name: connectionPoolSize
• Input Type: Numeric
• Description: Defines the number of connections that can be open simultaneously for interactions with the database.
• Default Value: 10
• Effects of Change: Increasing this value can improve throughput under heavy load, allowing more simultaneous requests to be handled, while lowering it can conserve resources and avoid database overloading.

4. Enable Retries

• Name: enableRetries
• Input Type: Boolean
• Description: A toggle that determines whether failed queries should be automatically retried based on predefined logic.
• Default Value: false
• Effects of Change: Enabling retries may enhance reliability in environments where transient failures occur frequently, but it can also lead to increased latency as failed requests are retried.

5. Logging Level

• Name: loggingLevel
• Input Type: Dropdown
• Description: Specifies the verbosity level of logs generated during operations. Options typically include none, error, warning, info, and debug.
• Default Value: info
• Effects of Change: Lowering the logging level to none conserves storage and reduces noise in logs, while a detailed debug level increases verbosity, aiding in troubleshooting but may clutter log outputs.

How It Works

The index integration adapts to various database systems by dynamically loading the necessary database adapter based on user-defined configurations. By extending support for both mandatory and optional database libraries, the integration focuses on delivering the required functionalities for data management while maintaining performance. Internally, it utilizes appropriate database query patterns tailored to the connected data source, ensuring optimal data retrieval and integrity.

Upon initialization, the integration reads the configuration settings established by the user and establishes connections to the selected database. Using async operations, queries are executed within defined timeouts, and results are returned to the calling services or components within Vantage.

Expected Data

The index integration expects input data in a structured format appropriate for each database type. For instance:

• For Firestore: A data structure representing documents and collections.
• For MongoDB: BSON objects for CRUD operations.
• For DynamoDB: Attribute-value pairs for manipulation.
• For PostgreSQL: SQL queries formatted correctly based on relational model conventions.

Each data interaction should conform to the selected database's schema and comply with its operational constraints.

Use Cases & Examples

Use Case 1: E-commerce Analytics

An e-commerce platform needs to aggregate sales data from multiple data sources (MongoDB for product information and PostgreSQL for user transactions) to generate comprehensive sales analysis and reporting. The integration enables seamless interactions with both databases, allowing real-time data insights.

Use Case 2: Multi-Cloud Data Management

A company operates across different cloud providers, utilizing Firestore for user profiles and DynamoDB for inventory management. This integration allows it to manage data efficiently across platforms without the need to rewrite application logic.

Configuration Example

Here’s how the index integration might be configured for the multi-cloud data management use case:

{
  "databaseAdapter": "dynamo",
  "queryTimeout": 20000,
  "connectionPoolSize": 15,
  "enableRetries": true,
  "loggingLevel": "info"
}

In this example configuration:

• databaseAdapter is set to dynamo for handling DynamoDB interactions.
• queryTimeout is limited to 20 seconds to minimize wait times.
• connectionPoolSize is increased to 15 to handle increased requests effectively.
• enableRetries is enabled to ensure queries are retried upon transient errors.
• loggingLevel is set to info to strike a balance between detail and verbosity in logs for operational insight.

This configuration allows the organization to effectively manage its data across multiple cloud environments while ensuring performance and reliability.

AI Integrations and Billing Impact

The index integration is designed to function autonomously without direct AI functionalities embedded within it; however, it can serve as a backend support for AI-driven analytics components built on Vantage. The performance metrics collected through the integration can inform machine learning models if utilized in conjunction with Vantage’s analytics features.

Regarding billing, using this integration to connect with various databases generates costs based on the database interactions defined in the user’s pricing plan. High query volume and increased connection pool size may lead to higher costs depending on the cloud provider's pricing model relevant to the databases in use. Users should monitor their configurations to optimize performance and manage costs effectively.