Skip to content

Datapool

Go to Datapool

The Datapool is a module of the BotCity Orchestrator that allows for efficient management of batch item processing.

Through the Datapool, you can have control and granularity over the queue of items that need to be processed, making it an essential resource for automations that process a large volume of data.

Screenshot of a Datapool overview. At the top, general information: Datapool details including its status, total number of items, average time per item; Items in Processing including Pending Items, Processed Items, and Completed Items; Exception cases including Error Items, Items on hold, and Cancelled Items. Below, search bar, filters, and Action buttons: Add Items, Import CSV, and bulk actions. At the bottom is the Datapool Items section, displaying a table with the fields Entry, Priority, Status, Termination Message, Error Type, Auto-retry, Processing Time, Lifecycle of each item.

Overview

Key Features

  • Task Triggering: Automatic task triggering for execution, based on the creation of new items in the queue.

Key Benefits

  • Complexity Abstraction: Eliminates the need to implement complex logic in the code, as it has native handling for concurrent consumption and reprocessing scenarios.
  • Centralized Management: Offers centralized management directly in the BotCity Orchestrator, facilitating control and monitoring, and is already integrated with other platform resources. Eliminates the need to depend on external databases and auxiliary tools.
  • Simplified Integrations: You can read from any database and transform the data into items in the Datapool, adding them to the processing queue through APIs, SDKs, and .csv files. Easily extract processing data and integrate with your data visualization platforms.

Use Cases

  • Batch Item Processing: Ideal for automations that need to process a large volume of data or batches of items, such as registrations, updates, and queries.
  • Producer/Consumer Scenarios: An excellent alternative for scenarios where one automation generates demand by creating new items in the Datapool, and another automation consumes the queue and performs the processing flow of these items.