Close popover

Table of Contents

Configuring the Data Flow service

Version:

In the Data Flow service, you can run data flows in batch mode or real time (stream) mode. Specify the number of Pega Platform threads that you want to use for running data flows in each mode.

This procedure applies only to on-premises deployments.
Assign the Stream and Batch node types to Pega Platform nodes. To scale the Data Flow service horizontally, assign the corresponding node type to a higher number of nodes.

For more information, see Assigning node types to nodes for on-premises environments.

  1. In the header of Dev Studio, click Configure Decisioning Infrastructure Services Data flow .

  2. In the Service list, select the node types for which you want to configure the number of threads.

    Batch nodes process batch data flow runs. Real-time nodes process streaming data flows.
  3. In the Data flow nodes section, click Edit settings.

  4. In the Thread count field, enter the number of threads that you want to use for running data flows in the selected mode.

    To scale the Data Flow service vertically, increase the current number of threads.
    If you divide the source of a data flow into five partitions, Pega Platform divides the data flow run into five assignments, and then processes the assignments simultaneously on separate threads, if five threads are available.

    Pega Platform calculates the number of available threads by multiplying the thread count by the number of nodes. For example, with two nodes and the thread count set to 5, the data flow run uses five threads and five threads remain idle.

  5. Click Submit.

  • Managing decision management nodes

    Manage the decision management nodes in your application by running certain actions for them, for example, repair or clean-up.

  • Enabling decision management services

    Decision management services comprise the technical foundation of decision management. Learn more about decision management services and how to enable them to fully benefit from next-best-action strategies and other decision management features in Pega Platform.

  • Status of decision management nodes

    For each decision management node on the Services landing page, the Status column displays the current state of a selected node. Use the following reference information to better understand the status of decision management nodes in your application.

  • Data Flows landing page

    This landing page provides facilities for managing data flows in your application. Data flows allow you to sequence and combine data based on various sources, and write the results to a destination. Data flow runs that are initiated through this landing page run in the access group context. They always use the checked-in instance of the Data Flow rule and the referenced rules.

Have a question? Get answers now.

Visit the Collaboration Center to ask questions, engage in discussions, share ideas, and help others.