Table of Contents

Filtering incoming data


Only available versions of this content are shown in the dropdown

Filter incoming data to reduce the number of records that your data flow needs to process. Specify filter conditions to make sure that you get the data that is applicable to your use case. Reducing the number of records that your data flow needs to process, decreases the processing time and hardware utilization.

  1. In a data flow, click the Plus icon on a shape, and select Filter.

  2. Double-click the Filter shape to configure it.

  3. In the Name field, enter a name for the shape.

  4. In the Filter conditions section, click Add condition.

  5. In the left field, enter the name of a property that is evaluated by the filter.

  6. From the dropdown list, select a comparison operator.

  7. In the right field, enter a value for your filter condition.

  8. Click Add condition to add more conditions.

A data record that enters the Filter shape is compared against the filter conditions. When the record matches the conditions, the Filter shape outputs the record for further processing in the remaining data flow shapes. For example, to select customers who are older than 18 years old and are employed, your filter conditions can look like this: .CustomerAge >= 18.IsEmployed = true

The following figure shows an example filter configuration:

strategyFilteringIncomingData icon

Related Content

Have a question? Get answers now.

Visit the Collaboration Center to ask questions, engage in discussions, share ideas, and help others.