Support Article
BIX extract rule caused a full load
SA-80629
Summary
The Business Intelligence Exchange (BIX) Extract rule is configured with the 'Use last updated time as start' filter criteria checked.
The Extract run retrieves all the records from the database. The Extract rule is executed by setting the below parameters and running the pxExtractDataWithArgs activty.
param.pyExtractName
param.pyExtractClassName
param.pyArgs default to -c
BIX extract rule caused a full load.
Error Messages
Not Applicable
Steps to Reproduce
- Configure an Extract rule with the 'Use last updated time as start' filter criteria checked.
- Execute the BIX.
Root Cause
The 'Use last updated time as start' checkbox used the pxCommitDateTime in the filter. Prior to Pega 7.1.8, there was just the pxUpdateDateTime filter. However, now there are also pxSaveDateTime and pxCommitDateTime which are updated at different times.
The pxUpdateDateTime filter is only set by specific activities in the rules layer (for example, a simple Obj-Open, Property-Set, and Obj-Save would not update this timestamp).
The pxSaveDateTime is always set by the engine layer when the BLOB is updated (for example, any Obj-Save is recorded here).
The pxCommitDateTime is always set by the engine layer when the row is updated (for example, imports and expose are recorded here).
During normal flow processing when a case is updated, pxUpdateDateTime, pxSaveDateTime, and pxCommitDateTime properties are updated since the flow updates pxUpdateDateTime. The OBJ-save updates the BLOB and thus pxSaveDateTime and then the row in the database is updated and thus pxCommitDateTime is updated.
Tools such as the Column Populator do not make changes to the pxUpdateDateTime property or the BLOB itself. Hence, pxSaveDateTime is not updated. However, each row in the table is updated to populate the exposed columns and thus pxCommitDateTime is also updated.
Resolution
To obtain only the updates to the work objects, change the Extract filter to use the symbolic 'Last Extraction Time' based on any DateTime property. pxUpdateDateTime or pxSaveDateTime can also be used.
Refer to the BIX User's Guide for more information on this Symbolic value: https://community.pega.com/knowledgebase/documents/bix-722-user-guide
Published August 15, 2019 - Updated October 8, 2020
Have a question? Get answers now.
Visit the Collaboration Center to ask questions, engage in discussions, share ideas, and help others.