Support Article
Filter & Sorting issue with Huge volume of data
SA-5205
Summary
In a repeat grid which refers to a report, the column filter function will show a pop up which will then show a list of unique values of the column. The filter is only showing unique values of the first 1000 records of the report. This causes data discrepancies. As for the filter list sorting, whilst it displays the unique values in a sorting mode as that of the column, it does not display the values in the order of occurence when the data set is orted on other columns.
Error Messages
No error message. Observed an undesired behavior where the column filter list incomplete unique value, taking only from the first 1000 records. It also did not display the values in the order of occurence when the data set is sorted on other columns.
Steps to Reproduce
- Populate a record set to have more than 1000 records, e.g. 1005 records.
- Create a report definition to fetch those records.
- Run the report definition. Open the first record and update a property (e.g. note) to a certain value. Save it.
- Navigate to the last page. Open the last record and update the same property to a different value. Save it.
- Set a repeat grid that displays the report definition with the updated property (e.g. note) displayed as one of the column.
- Try apply filtering on the property.
- Observe that only the first value is listed. This is because the 2nd value is outside the first 1000 records.
- Try sorting columns other than that is filtered.
- Observe the unique value is not displayed in the order of occurence as that in the repeat grid.
Root Cause
The root cause of this problem is a defect in Pegasystems’ code/rules. A value of 1000 is hardcoded in pzGetGridColUniqueValues as the pyMaxRecords when the activity builds the unique values for the filter. The sorting is also taking in the sorting of the filtered column.
Resolution
This issue is resolved by hotfix item HFIX-20232 which is to enable filter to display 1000 unique values instead of values from the first 1000 records.
As for the sorting, the consideration was that taking in the current sorting of the repeat grid will have an undesired performance impact to the system. However, as this is a functionality that is looked for by the the user, hence an enhancement request FDBK-9999 was created.
Published January 31, 2016 - Updated October 8, 2020
Have a question? Get answers now.
Visit the Collaboration Center to ask questions, engage in discussions, share ideas, and help others.