SA-2826

Summary

File Listener is configured for "Asynchronous (queue for execution) mode".  When processing each row, it is possible to have application specific validation for each row. This validation can look at various aspects of the file data (eg to ensure the headers are entered correctly, date fields are populated in the correct format, mandatory columns are populated etc.). When one of these conditions is not met, the Listener activity will sett Page Messages and calling the Activity-Set-Status as follows:
Severity: Fail
Message: param.ErrorMessage (param.ErrorMessage contains the text for the appropriate issue)

The result of this is that the Queue item will contain the error message and exceptions. The expectation is that the queue item to be retried and moved to Broken-Process Item status after 5 unsuccessful attempts.

The ItemStatus is "Scheduled" and the pyAttempts is 1. These queue items do not get processed again and the queue instances remain.

Error Messages

NA

Steps to Reproduce

NA

Root Cause

When Asynchronous (queue for execution) mode is selected for the Rule-Service-File, the processing of records (or of batches of records) happens
in the following order:

1. The listener thread reads a record (or a batch of records), creates a queue item containing the retrieved data and stores it as a
queue item in the PRPC DB queue.
1. The listener uses engine API to create a background thread and hands over the queue item to this thread for processing.
2. The listener thread repeats steps 1 and 2.

Multiple batch requestors running in separate background threads concurrently.
If an execution of any of background threads fails and the Service Request Processor configuration warrants a retry, the queue item stored
in the queue changes its status to "Scheduled" to be picked up and processed by an Agent that monitors this queue. When the Agent wakes up and there is a
number of queue items in the queue that are marked for a scheduled execution, the Agent thread processes them sequentially.
When you have DEBUG level set for class com.pega.pegarules.session.internal.engineinterface.service.RequestProcessorImpl, you can see in the PRPC log a message "Executing 'processQueue' for...". that indicates the start of the processing of the queue by the Agent.

Resolution

Configure multiple Agents monitoring the same queue if a concurrent processing of the queue is desired.

Do not confuse this mechanism of handling asynchronous execution failures with the File Listener's Error Recovery, which in this mode deals with file reading failures. If an error occurs while File Listener reads a file and the Error Recovery is enabled, this file will be read again and, if successful, it will be processed in the Asynchronous mode as described above.

For PRPC 6.3sp1, the Pega-IntSvcs agent has an entry which does the processing for the default queue class:

The Class name is: "System-Queue-ExecutionRequest-Service-Default"

This is also the class that is used by the Request Processor instance configured for your Listener.

Also, you could create multiple agent entries in order to allow for multi-threaded processing of these retries, if you feel that you need to have this feature. The agents will each use the prpc-provided queue iterator code (which utilized the stored proc sppr_sys_reservequeueitem_b) to lock and retrieve entries from the shared queue.