awsGetObjectInfo

Purpose

This node is designed to retrieve metadata information about an object stored in an Amazon S3 bucket. By integrating with the AWS S3 service, this component allows users to obtain detailed insights into the specific object, including its size, last modified date, content type, and storage class. It serves as a utility for applications that require information about files stored in S3, enabling better management and understanding of cloud-stored data.

Settings

This node accepts various inputs and configuration settings. Below is a detailed breakdown of each setting:

1. bucket_name

• Input Type: String
• What it Does: This setting specifies the name of the S3 bucket where the target object resides. If this input is not provided, the logic will attempt to fall back to the configured value.
• Default Value: Empty string. If neither the input nor the configuration value is set, the function will throw an error.

2. object_key

• Input Type: String
• What it Does: This setting denotes the key (path and filename) of the object within the specified S3 bucket. Similar to bucket_name, if this input is not provided, the function will use the configuration value.
• Default Value: Empty string. As with bucket_name, if both are absent, an error will be raised indicating the necessity of this field.

Context

• Input Type: Object
• What it Does: While not explicitly defined in the settings, the context is typically used to pass environment-specific information and credentials required for building the connection instance. This context enables secure access to AWS services.
• Default Value: Context object passed by the function caller, which is expected to contain necessary configurations.

How It Works

1. Integration Setup: The function begins by attempting to create a connection instance with AWS S3 using a secure connection. The context provided is important to establish this connection securely.

2. Parameter Preparation: It prepares an object, params, which incorporates the bucket_name and object_key. The function first checks if these parameters are provided directly as inputs. If they aren't, it defaults to checking the configuration.

3. Validation: The function checks if both the bucket_name and object_key are non-empty. If either is missing, an error is thrown indicating that this information is required.

4. Object Metadata Retrieval: Once the parameters are validated, the function calls the getObjectInfo method of the integration instance, passing in the params. This retrieves the object metadata from S3.

5. Output Preparation: The information retrieved is structured into an output object, including details such as the object's size, last modified date, content type, storage class, and the retrieval timestamp.

6. Error Handling: If any errors occur during the process, they are logged, and a generic error message is returned, ensuring that the calling function can handle errors gracefully.

Data Expectations

• Inputs:
  • bucket_name: Name of the S3 bucket (mandatory).
  • object_key: Key of the object in the S3 bucket (mandatory).
• Output: An object containing detailed information about the specified S3 object, which includes:
  • object_info: Detailed metadata of the object.
  • bucket: The name of the bucket.
  • key: The object key.
  • size: Size of the object in bytes.
  • lastModified: Timestamp of the last modification.
  • contentType: MIME type of the object.
  • storageClass: Storage class of the object.
  • meta: Contains retrieval timestamp.

Use Cases & Examples

Use Cases

1. File Management Systems: A business could use this functionality within a file management system to allow users to view metadata about files stored in S3, helping them understand file sizes and types for better organization.

2. Data Analytics: Data analysts could utilize this component to gather information about files that are being processed or analyzed, allowing them to make informed decisions based on file size and type.

3. Reporting Applications: A reporting tool could integrate this logic to generate reports based on the size and storage class of different S3 objects, helping users to optimize their cloud storage costs.

Example Configuration

Use Case: A data analytics team needs to retrieve the metadata of a large dataset stored in S3 for analysis.

Sample Configuration Data:

{
  "inputs": {
    "bucket_name": "my-dataset-bucket",
    "object_key": "large_dataset.csv"
  },
  "config": {
    "bucket_name": "default-bucket", // This value will be ignored since inputs are provided.
    "object_key": "" // No fallback will occur since this is an input.
  },
  "context": {
    // context would include necessary AWS credentials and configurations.
  }
}

In this example, the function will successfully retrieve the metadata for the large_dataset.csv object stored within my-dataset-bucket. If the object exists, the output will include its size, last modified date, and other relevant information to enable effective data analysis tasks.