Creating a data flow

Create a data flow to process and move data between data sources. Customize your data flow by adding data flow shapes and by referencing other business rules to do more complex data operations. For example, a simple data flow can move data from a single data set, apply a filter, and save the results in a different data set. More complex data flows can be sourced by other data flows, can apply strategies for data processing, and open a case or trigger an activity as the final outcome of the data flow.

1. In the header of Dev Studio, click Create > Data Model > Data Flow.
2. In the Create Data Flow tab, create the rule that stores the data flow:
   1. In the header of Dev Studio, click Create > Data Model > Data Flow.
   2. On the Create form, enter values in the fields to define the context of the flow.
   3. In the Label field, describe the purpose of the data flow.
   4. Optional: To change the default identifier for the data flow, click Edit, enter a meaningful name, and then click OK.
   5. In the Apply to field, press the Down arrow key, and then select the class that defines the scope of the flow.
      The class controls which rules the data flow can use. It also controls which rules can call the data flow.
   6. In the Add to ruleset field, select the name and version of a ruleset that stores the data flow.
   7. Click Create and open.
3. In the Edit Data flow tab, double-click the Source shape.
4. In the Source configurations window, in the Source list, define a primary data source for the data flow by selecting one of the following options:
   • To receive data from an activity or from a data flow with a destination that refers to your data flow, select Abstract.
   • To receive data from a different data flow, select Data flow. Ensure that the data flow that you select has an abstract destination defined.
   • To receive data from a data set, select Data set. If you select a streaming data set, such as Kaska, Kinesis, or Stream, in the Read options section, define a read option for the data flow:
     • To read both real-time records and data records that are stored before the start of the data flow, select Read existing and new records.
     • To read only real real-time records, select Only read new records.

     For more information, see Data Set rule form - Completing Data Set tab.

   • To retrieve and sort information from the PegaRULES database, an external database, or an Elasticsearch index, select Report definition.
   Note: Secondary sources appear in the Data Flow tab when you start combining and merging data. Secondary sources can originate from a data set, data flow, or report definition.
5. In the Source configurations window, click Submit.
6. Optional: To facilitate data processing, transform data that comes from the data source by performing one or more of the following procedures:
   • Filter incoming data
   • Combine data from two sources into a page or page list
   • Convert the class of the incoming data pages
   • Merge data
7. Optional: To apply advanced data processing on data that comes from the data source, call other rule types from the data flow by performing one or more of the following procedures:
   • Apply complex data transformations
   • Apply event strategies for complex event processing
   • Apply strategies for improving results of customer interactions
   • Apply text analysis on the records that contain text
8. In the Edit Data flow tab, double-click the Destination shape.
9. In the Destination configurations window, in the Destination list, define the output point of the data flow by selecting one of the following options:
   • If you want other data flows to use your data flow as their source, select Abstract.
   • If you want an activity to use the output data from your data flow, select Activity.
   • If you want to start a case as the result of a completed data flow, select Case. The created case contains the output data from your data flow.
   • If you want to send output data to a different data flow, select Data flow. Ensure that the data flow that you select has an abstract source defined.
   • To save the output data into a data set, select Data set.
     Note: Do not save data into Monte Carlo, Stream, or social media data sets.

     For more information, see Data Set rule form - Completing Data Set tab.

10. In the Source configurations window, click Submit.
11. In the Edit data flow tab, click Save.