Creating a batch run for data flows

You can create batch runs for data flows that do not have a Stream data set as the primary input. Data flow runs that are initiated through the Data Flows landing page process data in the access group context. They always use the checked-in instance of the data flow rule and the referenced rules.

1. Click Designer Studio> Decisioning > Decisions > Data Flows > Batch processing.
2. In the Batch processing tab, click New.
3. Specify the Applies To class, the data flow name, and the access group.
4. Optional: Click the Advanced section.

   1. Specify a pre-processing activity that you want to run before running the data flow.

   2. Specify a post-processing activity that you want to run after running the data flow.

      Each activity is located in its Applies To class.

   3. Select the Distribute run across multiple nodes check box.

      Note: This option is available when the primary source of your data flow is configured with a partitioning key. For details, see the Related information link about the data sets.

5. Optional: For data flows that contain an event strategy rule, configure how to save the state of your data flow work items in the State Management section:
   1. Specify the persistence type:
      • Memory - This persistence type keeps the event strategy state in running memory and writes the output to a destination when the data flow finishes running. The data is processed faster, but it can be lost if a system failure occurs.
      • Database - This persistence type periodically replicates the state of an event strategy to the Cassandra database located in the Decision Data Store and stores it in the form of key values. When you select this type of data persistence, if a system failure occurs, you can fully restore the state of the event strategy and continue processing data.
   2. Specify the frequency for storing the state of the event strategy:
      • Enter the elapsed time (in seconds) between the consecutive event strategy store operations.
      • Enter the number of events between the consecutive event strategy store operations.
   3. In the Target cache size field, specify the maximum size of the cache for the state management data. The default value is 10 megabytes.
6. Click Done.
7. In the Run details window that opens, click Start to run your data flow. The Run details window contains the progress and statistics of your data flow work item.

Open topic with navigation