Loading pxActuals data into Visual Business Director might cause high memory utilization
High memory utilization and an out of memory error might occur in production environments that run Pega® Visual Business Director for business reporting. This memory issue might occur after a planned node restart or when a query is submitted to Visual Business Director, and Pega Platform attempts to retrieve Interaction History (IH) records from the pxActuals data set and load those records into Visual Business Director for business reporting.
The root cause of this issue is physical memory overload when Pega Platform, as a result of a node restart or a query to Visual Business Director, attempts to retrieve a number of Interaction History records from the pxActuals data set for business reporting, and that number exceeds the memory capability of the node that runs the Visual Business Director service. Memory issues can occur for the following types of Visual Business Director-related system events:
- Queries that cover a specific date or time period and result in the retrieval of a substantially sized partition of Interaction History data
- Queries without a specified time range requirement for retrieving Interaction History data (all available Interaction History data is loaded)
- Restart of a Pega Platform node that is part of the Visual Business Director service, with the Eager load mode enabled (all available Interaction History data is loaded)
- Rerunning a past Pega Marketing campaign (for example, if you rerun a campaign that was first run three years ago, Pega Platform attempts to retrieve all Interaction History records that have been captured since the initial campaign run)
- Visual Business Director caching of a compressed version of Interaction History that contains all columns in the IH Fact table and IH Dimension tables. Application-specific columns that are added to the IH tables result in more data being cached in Visual Business Director
The hotfixes that have been released for Pega Platform 7.2 and later greatly reduce the amount of memory that is required to run Visual Business Director by automatically filtering out columns that are not relevant to Visual Business Director reporting. For more information, contact Pega Global Customer Support.
The processing power that is required to retrieve large amounts of VBD data can strain multiple application resources. For example, assume that you have an IH Fact table that stores 100 million Actuals records and that the VBD server is configured to retrieve that data in the Lazy load mode (the server retrieves the data only from the time period that is specified in each query). In this setup, the VBD loads Actuals by running one database query each day. If the time period for retrieving the Actuals data generates multiple database queries, a substantial amount of data will be loaded into the memory of the VBD server. This might cause an overload of the VBD server memory and CPU, database server CPU, and the network.
Additionally, each time you access the Actuals data source from the Pega Platform, the VBD server loads all the data, provided that no start date is specified for loading the data on the Visual Business Director landing page. If the Actuals data source consists of a large number of records (for example, 100 million) and no start date for loading data is specified, resource consumption might increase and system response times might slow down.
You can limit the number of VBD queries by setting the start date for loading the Actuals data. This way, you can make the VBD data retrieval stay within your acceptable limits.
Solution: Decrease the memory requirements
You can decrease the amount of physical memory that Visual Business Director requires to operate.
- Apply the Pega Decision Strategy Manager (DSM) cumulative hotfix that, among other enhancements, significantly decreases the amount of memory that is required to run Visual Business Director. For more information about the DSM cumulative hotfix, contact Pega Global Customer Support.
- Limit VBD memory consumption by specifying the start date for data retrieval in your application.
- Increase the amount of physical memory that is available to your application.
Solution: Limit the number of resources required for VBD
If you are not using the VBD Planner in your application, for example, because you are using only the core decision management functionality as part of the Pega-provided DMSample application, you can limit the number of resources allocated to the VBD server. For example, you can configure the Visual Business Director service to run on a node that is assigned to the Data Flow service or that processes data as part of any inbound decisioning functionality.
For information on configuring Pega Platform nodes as part of decisioning management, see the Services landing page.
To further reduce Visual Business Director memory consumption, you can control the amount of Interaction History data that is loaded by the pxActuals data set by setting a start date for data retrieval. You can set a date from the recent past or to a future date to disable all data retrieval. By doing so, no or only some Interaction History data will be loaded into Visual Business Director, regardless of the time range requirement that is specified in Visual Business Director queries, or Eager load mode node restarts. The Start date setting corresponds to the pxOutcomeTime property of Interaction History records in the pxActuals data set.
- In Designer Studio, click Decisioning > Monitoring > Visual Business Director > Data Sources.
- On the Data Sources tab of the Visual Business Director landing page, click the Expand icon next to the Actuals data source.
- In the Start Date field, click the Calendar icon and select the start date for retrieving Interaction History records from the pxActuals data set.
- Click Save.
Setting the start date for loading Actuals into Visual Business Director