SA-2349

Summary

Issue (question) relates to a File Listener which has a requirement to process a large number of records in a csv.

Error Messages

NA

Steps to Reproduce

Execution Mode: Execute asynchronously (queue for execution)
Requests per Queue Item: 1
Processing Method: record at a time

In the Request Processor, configured the following:
Maximum number of execution attempts: 5
Keep in queue after all execution attempts failed : true
Keep in queue after successful Execution: false

When a file is placed for processing:
1) PRPC starts processing 1 record at a time
2) PRPC creates Queue instances as configured in the Request processor ( instances of System-Queue-ExecutionRequest-Service-AppBulkQueue)
3) PRPC creates multiple Application requestors (30) and quickly processes the queued instances. Processing is fast and multiple records get processed due this
4) As part of the processing, we are creating and updating work objects. Hence there are scenarios where the

Request activity may fail (Work Object held by other requestor, validation errors etc). In this scenario, PRPC throws the exception, increments the pyAttempts value and places the queue item in "Scheduled" status. This is the expected behavior since we have configured Maximum number of attempts to 5 in the Service Request Processor. Now when the Scheduled Items are processed (re-queued items), we are finding that only 1 Application requestor is created and the processing of the remaining queue items is very slow. PRPC must create the same (30) number of Application requestors and process the re-schdeduled rows as well

Root Cause

This issue relates to the methods used to process the initial and re-queued items.When Asynchronous (queue for execution) mode is selected for the Rule-Service-File, the processing of records (or of batches of records) happens in the following order:

 

1. The listener thread reads a record (or a batch of records), creates a queue item containing the retrieved data and stores it as a
   queue item in the PRPC DB queue.
2. The listener uses engine API to create a background thread and hands over the queue item to this thread for processing.
3. The listener thread repeats steps 1 and 2.

 

This results in multiple batch requestors running in separate background threads concurrently.

 

If an execution of any of background threads fails and the Service Request Processor configuration warrants a retry, the queue item stored in the queue changes its status to “Scheduled” to be picked up and processed by an Agent that monitors this queue. When the Agent wakes up and there are a number of queue items in the queue marked for a scheduled execution, the Agent thread processes them sequentially. You can see in the PRPC log a message “Executing 'processQueue’ for…”. that indicates the start of the processing of the queue by the Agent.

Resolution

 

One may consider configuring multiple Agents monitoring the same queue if a concurrent processing of the queue is desired.