Optimizing BIX data extraction performance
Understanding performance enhancements of the Business Intelligence Exchange (BIX) configurations will help you to maximize the throughput of BIX extractions from your applications. Refer to the following table to incorporate enhancement considerations into your BIX extraction process as needed.
| Location of BIX improvement | Performance enhancement consideration |
|---|---|
| BIX logs | Setting BIX logging to a level of DEBUG or ALERTS can cause performance issues. |
| XML-based extractions occur much faster than CSV-based extractions or database schema extractions because data in embedded page lists and page groups does not require normalization in XML as it is being written. | |
| Configuring fewer properties for BIX extractions and configuring properties that are not nested within a class structure increases extraction performance. | |
| Running multiple BIX extractions with different filter conditions that use the same class in parallel enhances performance. For example, using different filter criteria for separate product lines, geographies, case resolutions, or organizational divisions. | |
| Enabling incremental BIX extractions limits the properties extracted to those
with an updated pxCommitDateTime instance when compared to the
last extraction to increase subsequent BIX data extraction performance. Properties
with an updated pxCommitDateTime include classes created or
changed by an operator or automatically by the application. Note: The BIX extraction
process skips properties that contain pxCommitDateTime
instances with a null value including child classes
incorporated into the extraction by the -c parameter in the
Job Scheduler ruleform.
|
BIX high-throughput data extractions
Pega Platform uses high-throughput data extractions for CSV extractions for specific databases to improve performance of the BIX extraction process. BIX high-throughput data extractions occur automatically under the following extraction process conditions:- The BIX extraction process occurs from BLOB-less class data. For details see Optimizing properties from the user interface.
- The BIX extraction process occurs from an external Postgres database. For details see Creating database instances for JDBC URLs.
- The BIX extraction process uses a CSV format. For details, seeCreating and running an Extract rule.
When you use the high-throughput data extraction option, the following functionality is not available:
- You cannot input new property names for the CSV files when creating the Extract rule.
- You cannot append the -c parameter to include child classes.
- You cannot append the -x parameter to include header and footer information in the output. For details see BIX parameters and command-line arguments and Using the Job Scheduler rule to extract data with BIX.
- You cannot use the checksum in the manifest summary. For details see Creating and running an Extract rule.
Create a dynamic system setting with keystring bix/useHighThroughputDownloadForCSV and set value field to false to disable high-throughput extractions for CSV extract files from an external Postgres databases with BLOB-less class data.