Skip to main content
LinkedIn
Copied!

Table of Contents

Monitoring predictions

Analyze how successful your predictions are in predicting the outcomes that bring value to your business. Gain insights by reviewing performance charts for predictions and the models that drive them.

This procedure applies to predictions that are used in customer engagement and case management. For information about analyzing text predictions, see Monitoring text predictions.

If a prediction uses more than one model, you can select the outcome for which you want to see performance statistics, for example, clicks, conversion, or clicks and conversion combined.

  1. In the navigation pane of Prediction Studio, click Predictions.

  2. From the list of predictions, open a prediction that you want to analyze.

  3. On the Analysis tab, click Prediction.

    1. In the Outcomes field, select the outcome to analyze.

      If a prediction predicts a single outcome, the outcome is already selected. If a prediction predicts outcomes that occur one after another, you can select each individual outcome or the two outcomes combined.
      For a prediction that predicts whether a customer who is likely to click a web banner is also likely to accept the corresponding offer and convert, you can select from the following outcomes:
      • Clicks
      • Conversion
      • Clicks + Conversion
    2. In the Time frame field, select the period that you want to analyze.

      The charts show the performance measures that are relevant to the selected outcome. For example, a performance analysis with regard to churn covers the churn rate, lift, AUC, and total cases, as in the following figure:

      Prediction analysis
      The Analysis tab in a prediction showing the performance
                                        charts for churn.
  4. Click the Models tab.

  5. If more that one model drives the prediction, in the Models field, select the model the performance of which you want to analyze.

    The performance chart shows how effective the model is over time. If a shadow model runs alongside an active model, the chart displays the performance curves for each model in the active-shadow pair. The comparison can help you determine which model is more effective and decide whether you want to promote the shadow model to the active model position. The analysis is available for both outcome-based and supporting models.

    The following figure shows the performance chart for the active model, Predict Churn DRF. This model is paired with a shadow model, Churn GBM. The chart shows two distinct lines that illustrate the performance of each model over time. Hover over the lines with your cursor to see the exact scores for each model at different times. In this example, the two lines converge around January 27, suggesting that the models performed equally well. The detailed information on mouse hover shows that the active model had a slightly higher score.

    A model performance chart in a prediction
    The Analysis tab in a prediction showing the performance of an
                                active model and its shadow as two distinct lines.
Did you find this content helpful?

Have a question? Get answers now.

Visit the Collaboration Center to ask questions, engage in discussions, share ideas, and help others.

Ready to crush complexity?

Experience the benefits of Pega Community when you log in.

We'd prefer it if you saw us at our best.

Pega Community has detected you are using a browser which may prevent you from experiencing the site as intended. To improve your experience, please update your browser.

Close Deprecation Notice
Contact us