Predict Action Propensity prediction strategy
PredictActionPropensity is a prediction strategy that is accessible for modification from Prediction Studio.
It starts by executing the OmniAdaptiveModel that is applicable to all actions and channels. The context for this model is issue, group, name, channel, and direction.
To improve the overall model performance, this strategy applies several techniques to the propensity calculation as described below.
Random control group
By default, 2% of the actions receive a random propensity between 0 and 1. A certain amount of randomness is always important to ensure the models continue to learn new patterns.
This is achieved by calculating a CustomerHash value between 0 and 100 based on the customer ID (pySubjectId) and the month name, then selecting 2% for a random Control group, and the remaining 98% for the Test group (using the Select Treatment Switch rule). Including the month name in the hash string provides each customer with a value that will remain constant for one month, so that the customer remains part of the same control group for that month.
Outbound model maturity
For outbound interactions, if the Outbound model maturity real-time control is set to true, the HandleModelMaturity sub-strategy is invoked.
The purpose of this strategy is to exclude actions (and treatments at a later stage) that have recently been introduced and for which there is little evidence on which the models can be trained.
Actions that are deemed to be already mature bypass this process through the Mature actions Exclusion rule. For actions that are less mature, an analytical method is used to filter out a portion based on their maturity (the less mature, the higher the proportion), and then the remaining actions have their propensity adjusted using Thompson sampling.
Finally, propensity smoothing is applied, resulting in AdjustedPropensity. Adjusted propensity uses a weighted average, taking into account StartingPropensity (at a weight of 1) and StartingEvidence (at a weight of 25) to ensure that new actions have a high initial propensity, which gives them a fighting chance against existing actions. After enough responses are received, the strategy shifts towards to using the model propensity.