Close popover

Table of Contents

Partition keys for Stream data sets

Version:

You can define a set of partition keys in a Stream data set to test how data flow processing is distributed across Data Flow service nodes in a multinode decision management environment by using the default load balancer. For example, you can test whether the intended number and type of partitions negatively affect the processing of a Data Flow rule that references an event strategy.

Create the partition keys in a Stream data set when your custom load balancer for Stream data sets is unavailable or busy, or in application environments where the production level is set to 1 - Sandbox, 2 - Development, or 3 - Quality assurance. If you set the production level to 4 - Staging or 5 - Production, then any Stream data set that has at least one partition key defined continues to process data, but is no longer distributed across multiple nodes. For more information on production levels, see Specifying the production level.

If the Stream data set feeds event data to an Event Strategy rule, you can define only a single partition key for that data set. That partition key must be the same as the event key that is defined in the Real-Time Data shape on the Event Strategy form. Otherwise, when you run the Data Flow, it fails.

An active Data Flow rule that references a Stream data set with least one partition key defined continues processing when nodes are added or removed from the cluster, for example, as a result of node failure or an intentional change in the node topology. However, any data that was not yet processed on the failed or disconnected node is lost.

  • Specifying the production level

    Define who can access and make changes to your system by specifying its production level.

  • Types of Data Set rules

    Learn about the types of data set rules that you can create in Pega Platform.

  • Event Strategy rule form - Completing the Event Strategy tab

    You use the Event Strategy tab to design and configure your event strategy components. A new instance of the Event strategy rule contains two shapes: Real-time data and Emit. You can add shapes by clicking the add icon that is available when you focus on a shape. To edit a shape, open the properties dialog box of a shape (by double-clicking the shape or by right-clicking and selecting the Properties option). The properties dialog box contains elements that are specific to a given shape.

  • Specifying system-wide security parameters by using system data instances

    Specify system-wide parameters, for example, the production level, to apply to all nodes in the system.

  • About Data Set rules

    Data sets define collections of records, allowing you to set up instances that make use of data abstraction to represent data stored in different sources and formats. Depending on the type selected when creating a new instance, data sets represent Visual Business Director (VBD) data sources, data in database tables or data in decision data stores. Through the data management operations for each data set type, you can read, insert and remove records. Data sets are used on their own through data management operations, as part of combined data

  • Data Set rules - Completing the Create, Save As, or Specialization form

Have a question? Get answers now.

Visit the Collaboration Center to ask questions, engage in discussions, share ideas, and help others.