Filter Logic Documentation

Overview

The "Filter" logic in the Vantage analytics and data platform is designed to process data arrays by applying specified logical conditions, allowing users to include or exclude certain data rows based on defined criteria. This logic is essential for refining datasets, enabling users to focus on relevant information and optimize their analytics.

Purpose

The primary purpose of the Filter logic is to enable dynamic filtering of data based on user-defined criteria. By allowing for a variety of comparison operators, the Filter logic empowers users to extract meaningful insights from large datasets by narrowing down their focus on particular subsets of data.

How It Works

The Filter logic processes an input array of data, applying a series of predicate functions defined by user-specified criteria. Each predicate function evaluates a specific condition for individual data rows. The final output consists only of the rows that satisfy all the defined conditions.

The functionality is encapsulated in two main parts:

1. Building predicates from criteria using the buildPredicateFromCriteria function.
2. Executing the filter logic with the execute method that filters data based on the constructed predicate.

Settings

Predicate

1. Name: predicate
2. Input Type: Function
3. Description: The predicate function is a user-defined function that takes in a row of data and returns a boolean indicating whether the row meets the filtering criteria. By providing a custom predicate, the user exhibits greater control over the filtering process.
4. Default Value: None (Must be provided, either as a function or through criteria).

Criteria

1. Name: criteria
2. Input Type: Object
3. Description: The criteria object defines conditions to build the predicate. Each key in the object represents a field of the data, and its value is another object containing operations (like eq, gt, etc.) mapped to the values to compare against. For instance, if a field has multiple operations (e.g., less than and greater than), both conditions must be true for the data row to be included.
4. Default Value: None (Must be provided if predicate is not defined).

Execution

• The filter’s execution method execute takes one input:
  • Input Data: Array
  • It filters the input data array based on the constructed predicate function derived from the given criteria or the provided predicate.

Error Handling

• The Filter logic must throw an error if neither a predicate nor criteria are provided, ensuring users are aware that a valid filtering mechanism must be established.

Data Expectations

The Filter logic expects an array of objects (each representing a data row) that matches the structure of the filtering criteria. Each object should contain keys corresponding to the fields specified in the criteria to allow for accurate comparisons.

Use Cases & Examples

Use Case 1: Customer Segmentation

A marketing team can use the Filter logic to segment customers based on their purchase history, including filtering customers who purchased more than $100 in the last month.

Use Case 2: User Behavior Analysis

A product team may want to analyze users who interacted with a specific feature of an application. By filtering user activity logs, they can identify how many users invoked that feature more than five times, leading to enhanced product development insights.

Use Case 3: Audit Log Monitoring

An IT department can utilize the Filter logic to monitor security logs, filtering out any unauthorized login attempts by setting criteria that signify failed login attempts from specific IP addresses.

Detailed Example

Use Case: Customer Segmentation

• Goal: Filter customers who have spent more than $100 in the past month.
• Sample Configuration Data:

const filterParams = {
  criteria: {
    purchaseAmount: {
      gt: 100
    },
    purchaseDate: {
      gte: '2023-09-01' // Assuming date format is in YYYY-MM-DD
    }
  }
};

// Creating filter node
const filterNode = createFilterNode(filterParams);

// Sample data array
const customerData = [
  { id: 1, name: 'Alice', purchaseAmount: 120, purchaseDate: '2023-09-05' },
  { id: 2, name: 'Bob', purchaseAmount: 80, purchaseDate: '2023-09-06' },
  { id: 3, name: 'Charlie', purchaseAmount: 150, purchaseDate: '2023-09-03' }
];

// Executing the filter
const filteredCustomers = filterNode.execute(customerData);
console.log(filteredCustomers); // Will log Alice and Charlie's data

In this example, only 'Alice' and 'Charlie' will be returned as they meet the filtering criteria of having spent greater than $100 in the specified date range.

AI Integrations

The Filter logic does not include specific AI integrations by default. However, it can be extended to work alongside machine learning models to dynamically filter datasets for training or inference, enhancing users' analytic capabilities by enabling real-time data adjustments.

Billing Impact

The use of the Filter logic itself does not have a direct impact on billing; however, the size of the data being processed may affect resource usage. Users should be aware that filtering large datasets may have performance implications, potentially leading to higher resource consumption, which in turn can influence billing for compute resources in the Vantage platform.