Skip to main content
LinkedIn
Copied!

Table of Contents

Configuring ethical bias checks

You can test your strategies for unwanted bias by creating an ethical bias policy. For example, you can test whether your strategies generate biased results by sending more actions to male rather than female customers, as well as check how big the bias is. Perform the following procedure to configure the fields that you want to use to measure bias, and to define the thresholds for detecting bias.

Assign the pzBiasPolicyConfiguration privilege to your access group. For more information, see "Optional: Granting permission to configure ethical bias policies" in the Pega Customer Decision Hub Implementation Guide on the Pega Customer Decision Hub product page.
  1. In the Pega Customer Decision Hub portal, click Configuration Ethical Bias Policy

  2. On the Bias fields tab, click Add bias field.

    Any properties that are related to your main customer class, such as gender, age, and ethnicity-related properties, are good candidates for bias testing, even if your strategies or AI models do not directly reference these properties. This is because indirect bias can still occur.
  3. If the property that you select has a number value, in the Add bias field window, specify whether you want Pega Platform to interpret the value as a category (for example, gender or ethnicity), or as an ordinal number (for example, for age).

    If there are many categorical values, the system only checks the 20 most frequent values for bias. In most cases, numerical values should not be classified as a category. However, if the property is not truly an ordinal number, you may want to treat it as a category, for example, you can use a postal code to test for location-related bias.
  4. On the Bias fields tab, review and configure the bias threshold settings for each issue in your business structure.

    The bias threshold measurement depends on the type of field that you select:
    • Rate ratio - Rate ratio is used to calculate bias for categorical fields by comparing the number of customers who were selected for an action to those not selected for an action, and then correlating the result to the selected bias field. For example, the rate ratio represented in the following table indicates that actions are sent more often to male rather than female customers:
      Female customers Male customers
      selected for the action 500 1000
      not selected for the action 20000 18000
      rate ratio 0.46 2.16
      A rate ratio of 1 represents no shifts in the distribution. You can select a warning range with a threshold between 0 (send a warning if any bias is detected), and 0.7 (send a warning only if very high bias is detected). You can also choose to ignore this bias field for a particular issue in your business structure. For each bias threshold setting, there is no difference made between a positive shift (towards > 1), and a negative shift (towards < 1)
      Bias threshold Allowed rate ratio range
      no bias allowed any bias with a 95% confidence for the rate ratio being greater than or less than 1 will be detected
      very light 0.9-1.11
      light 0.8-1.25
      heavy 0.66-1.50
      very heavy 0.5-2.0
      all bias allowed no bias detection

      For the rate ratio, the confidence interval is calculated using the approximation on the error on log on.

      Bias threshold values and allowed rate ratio range
      A diagram showing examples of measured rate ratios inside
                                        and outside of the allowed rate ratio range.
    • Gini coefficient - The Gini inequality coefficient is used to calculate bias for numerical fields. For example, it detects whether the distribution of age is different for customers who receive an action. A Gini coefficient of 0 represents no shifts in the distribution. You can select a warning threshold between 0 (send a warning if any bias is detected) and 0.50 - 2.00 (send a warning only if very high bias is detected). If the measured Gini coefficient is outside of the allowed range with a confidence interval of 95%, it will signify a significant bias. You can also choose to ignore this bias field for a particular issue in your business structure.
      For issues such as credit risk, set the value of the threshold lower than for marketing issues such as upsell, to better detect the variations.
      Bias threshold Allowed Gini range
      very light < 0.1
      very light < 0.2
      heavy < 0.5
      very heavy < 0.7
      no bias allowed any bias with a 95% confidence for the Gini coefficient to be greater than 0 will be detected
      all bias allowed no bias detection

      The confidence interval for the Gini coefficient is calculated using Delong's method.

      The confidence level for detecting bias above threshold can be set through a Dynamic System Setting (DSS). The default value is 0.9999 (or 99.99%). Lowering this level increases the probability of false alerts on bias detected.

      Dynamic System Setting for the confidence level used to determine the interval width

      Pega-DecisionEngine decision/simulation/ethicalbias/confidenceinterval 0.9999
  5. Click Save.

  6. To use the bias policy to test the behavior of your strategies:

    1. In the Pega Customer Decision Hub portal, click Simulation Testing.

    2. Create a new simulation test with the purpose Ethical bias.

Did you find this content helpful?

Have a question? Get answers now.

Visit the Collaboration Center to ask questions, engage in discussions, share ideas, and help others.

Ready to crush complexity?

Experience the benefits of Pega Community when you log in.

We'd prefer it if you saw us at our best.

Pega Community has detected you are using a browser which may prevent you from experiencing the site as intended. To improve your experience, please update your browser.

Close Deprecation Notice
Contact us