BIX extract rule caused a full load
The Business Intelligence Exchange (BIX) Extract rule is configured with the 'Use last updated time as start' filter criteria checked.
The Extract run retrieves all the records from the database. The Extract rule is executed by setting the below parameters and running the pxExtractDataWithArgs activty.
param.pyArgs default to -c
BIX extract rule caused a full load.
Steps to Reproduce
- Configure an Extract rule with the 'Use last updated time as start' filter criteria checked.
- Execute the BIX.
The 'Use last updated time as start' checkbox used the pxCommitDateTime in the filter. Prior to Pega 7.1.8, there was just the pxUpdateDateTime filter. However, now there are also pxSaveDateTime and pxCommitDateTime which are updated at different times.
The pxUpdateDateTime filter is only set by specific activities in the rules layer (for example, a simple Obj-Open, Property-Set, and Obj-Save would not update this timestamp).
The pxSaveDateTime is always set by the engine layer when the BLOB is updated (for example, any Obj-Save is recorded here).
The pxCommitDateTime is always set by the engine layer when the row is updated (for example, imports and expose are recorded here).
During normal flow processing when a case is updated, pxUpdateDateTime, pxSaveDateTime, and pxCommitDateTime properties are updated since the flow updates pxUpdateDateTime. The OBJ-save updates the BLOB and thus pxSaveDateTime and then the row in the database is updated and thus pxCommitDateTime is updated.
Tools such as the Column Populator do not make changes to the pxUpdateDateTime property or the BLOB itself. Hence, pxSaveDateTime is not updated. However, each row in the table is updated to populate the exposed columns and thus pxCommitDateTime is also updated.
To obtain only the updates to the work objects, change the Extract filter to use the symbolic 'Last Extraction Time' based on any DateTime property. pxUpdateDateTime or pxSaveDateTime can also be used.
Refer to the BIX User's Guide for more information on this Symbolic value: https://community.pega.com/knowledgebase/documents/bix-722-user-guide
0% found this useful