Close popover

Table of Contents

Filtering incoming data

Version:

Filter incoming data to reduce the number of records that your data flow needs to process. Specify filter conditions to make sure that you get the data that is applicable to your use case. Reducing the number of records that your data flow needs to process, decreases the processing time and hardware utilization.

  1. In a data flow, click the Plus icon on a shape, and select Filter.

  2. Double-click the Filter shape to configure it.

  3. In the Name field, enter a name for the shape.

  4. In the Filter conditions section, click Add condition.

  5. In the left field, enter the name of a property that is evaluated by the filter.

  6. From the dropdown list, select a comparison operator.

  7. In the right field, enter a value for your filter condition.

  8. Click Add condition to add more conditions.

A data record that enters the Filter shape is compared against the filter conditions. When the record matches the conditions, the Filter shape outputs the record for further processing in the remaining data flow shapes. For example, to filter out customers who are younger than 18 years old and are unemployed, your filter conditions can look like this: .CustomerAge > 18.IsEmployed = false

  • Creating a data flow

    Create a data flow to process and move data between data sources. Customize your data flow by adding data flow shapes and by referencing other business rules to do more complex data operations. For example, a simple data flow can move data from a single data set, apply a filter, and save the results in a different data set. More complex data flows can be sourced by other data flows, can apply strategies for data processing, and open a case or trigger an activity as the final outcome of the data flow.

  • Types of data flows

    Data flows are scalable data pipelines that you can build to sequence and combine data based on various data sources. Each data flow consists of components that transform data and enrich data processing with business rules.

Have a question? Get answers now.

Visit the Collaboration Center to ask questions, engage in discussions, share ideas, and help others.