writeFile Logic Documentation

Overview

The writeFile logic component is designed to facilitate the process of writing data to a file within a user's Intuidy folder on OneDrive. This component is essential for users looking to automate the storage of datasets or text output generated through various processes on the Vantage platform.

Purpose

The primary purpose of the writeFile logic is to enable users to export their datasets or text content directly into OneDrive, with options to specify the file format, naming convention, and organizational structure via subfolders. The logic encapsulates connections to OneDrive, ensuring that the necessary folders are created if they do not already exist.

Settings

The writeFile logic supports a variety of configurable settings that adjust its behavior. Below are the detailed settings:

1. fileName

• Type: String (required)
• Description: This setting specifies the name of the output file. If not specified, it defaults to export.csv. The logic appends a timestamp in ISO 8601 format to the filename to prevent overwriting existing files.
• Default Value: export.csv

2. mimeType

• Type: String (optional)
• Description: This setting defines the MIME type of the file being created. It helps to indicate the type of document being stored in OneDrive. If not specified, it defaults to text/csv.
• Default Value: text/csv

3. subfolder

• Type: String (optional)
• Description: This setting allows users to specify a subfolder within the "Intuidy" directory to save the file. If the specified subfolder does not already exist, the logic will be responsible for creating it.
• Default Value: None (no subfolder is created unless specified)

4. format

• Type: Dropdown (csv, json, text) (optional)

• Description: This setting determines the format in which the data is saved. The available options include:

  • csv: Comma-separated values, which is the default format.
  • json: JavaScript Object Notation format.
  • text: Plain text format.

  Changing this setting affects how the data is structured in the file.

• Default Value: csv

5. mode

• Type: Dropdown (auto, create, overwrite) (optional)

• Description: This setting specifies how the file should be handled if a file with the same name already exists:

  • auto: The system will automatically decide based on the situation.
  • create: It will create a new file regardless if an existing file with the same name is present.
  • overwrite: It will replace the existing file.

  The choice of mode impacts whether users risk losing previous data or maintaining existing files.

• Default Value: auto

Inputs

Required Input

• data: This input accepts either an array of objects or a string (raw text). The component expects this data as the primary content to be written to the file. If no valid data is provided, an error will be returned.

Outputs

Upon successful execution, the writeFile logic will return the following outputs:

• fileId: The unique identifier of the file created in OneDrive.
• fileName: The name of the created file, including timestamping.
• fileUrl: The web URL or link to access the file in OneDrive.
• mimeType: The MIME type of the created file.

How It Works

1. Data Input Handling: The logic first attempts to unwrap nested data structures provided by upstream logic components to extract the intended dataset.

2. Validation: It verifies that appropriate data is provided and returns an error if no data input is available.

3. Configuration Extraction: The logic then extracts and validates all configuration parameters such as filename, MIME type, subfolder, format, and mode.

4. Folder Management: It ensures that the necessary "Intuidy" folder exists, and creates a specified subfolder if needed.

5. Data Formatting: Based on the format setting, the input data is transformed into the corresponding string representation (CSV, JSON, or plain text).

6. File Writing: The data is finally written to a file in OneDrive, using parameters defined in the configuration.

7. Error Handling: The logic includes robust error handling mechanisms to manage integration issues with OneDrive and provide meaningful error messages in the event of failures.

Use Cases & Examples

Use Cases

1. Automated Report Generation: A business needs to generate and save daily reports of sales data that can be easily shared among team members. The writeFile logic can automatically compile this data and store it in a designated OneDrive folder.

2. Data Export for Analysis: A data engineer may need to export processed data from an ETL pipeline to share with other applications. Utilizing the writeFile component, this data can be exported in JSON format directly to the cloud for accessibility.

3. Backup Data Files: Users may require a backup of certain datasets at regular intervals. By configuring the writeFile logic to run on a schedule, they can automatically save copies of critical data sets without manual intervention.

Configuration Example

Use Case: Automating the backup of sales data.

Configuration:

{
    "config": {
        "fileName": "sales_data_backup.csv",
        "mimeType": "text/csv",
        "subfolder": "Backups",
        "format": "csv",
        "mode": "create"
    },
    "inputs": {
        "data": [
            { "transactionId": 123, "amount": 500, "date": "2023-10-01" },
            { "transactionId": 124, "amount": 750, "date": "2023-10-02" }
        ]
    }
}

Explanation: In this scenario, the writeFile logic is configured to create a file named sales_data_backup.csv in a subfolder called "Backups". It will write in CSV format, and since the mode is set to create, it will create a new file every time it runs, preserving all previous backups. The data input represents a list of transaction records that will be exported.