Support Article
FIle Listeners stop processing files and goes into sleep mode
SA-36517
Summary
User is using Pega 7.2.1 hosted on JBoss EAP 6.4.
They have several distinct file listeners, a couple of which use multiple concurrent threads.
It is frequently observed that several threads of these listeners stay in a "Sleeping" state, even though there are files present in the directory that they are listening to, and even though other threads of the same listener are actively processing files.
Some times they see 10/10 of file listener threads processing file and at other times they see only 1/10 or 2/10 of the file listener threads to be processing the files.
The Agent Threadpool Size is set to 15, and this behavior is observed with both pooled and non-pooled listeners.
The only way to force the listeners to start processing is to stop and start them, and even then they don't always see all of the threads start running.
Additionally, when a listener thread has stopped processing and they try clicking the "isAlive" button in the System Management Application (SMA), and the SMA returns the message that the selected listener is alive.
Error Messages
Not Applicable
Steps to Reproduce
- Start a multi-threaded file listener, and then regularly feed it batches of files to process.
- After a varying amount of time and a varying number of batches, several of the threads will stop processing, with "Last Access" dates in the past.
Root Cause
An issue in the custom application code or rules is identified as root cause of this issue.
Resolution
The following JBOSS tuning related changes in operating environment helped to resolve the issue:
1. Update standalone(-full).xml and change max-threads from default 10 to 30.
<remote connector-ref="remoting-connector" thread-pool-name="default"/>
<thread-pools>
<thread-pool name="default">
<max-threads count="30"/>
<keepalive-time time="100" unit="milliseconds"/>
</thread-pool>
</thread-pools>
2. Update standalone(-full).xml and change max-pool-size for slsb-strict-max-pool and mdb-strict-max-pool to 100 each. <pools>
<bean-instance-pools>
<strict-max-pool name="slsb-strict-max-pool" max-pool-size="100" instance-acquisition-timeout="5" instance-acquisition-timeout-unit="MINUTES"/>
<strict-max-pool name="mdb-strict-max-pool" max-pool-size="100" instance-acquisition-timeout="5" instance-acquisition-timeout-unit="MINUTES"/>
</bean-instance-pools>
</pools>
3. Update standalone(-full).xml and change the min-pool-size and max-pool-size of the AsyncConnectionFactory
<pooled-connection-factory name="AsyncConnectionFactory">
<transaction mode="xa"/>
<min-pool-size>40</min-pool-size>
<max-pool-size>100</max-pool-size>
<connectors>
<connector-ref connector-name="in-vm"/>
</connectors>
<entries>
<entry name="java:/jms/PRAsyncTCF"/>
</entries>
</pooled-connection-factory>
4. Update standalone(-full).xml and set the consumer-window-size to 10240 for the hornetq-ra connection factory.
<pooled-connection-factory name="hornetq-ra">
<transaction mode="xa"/>
<connectors>
<connector-ref connector-name="in-vm"/>
</connectors>
<entries>
<entry name="java:/JmsXA"/>
</entries>
<consumer-window-size>10240</consumer-window-size>
</pooled-connection-factory>
5. Update standalone(-full).xml and set the max-delivery-attempts to 1 and the redelivery-delay to 0.
<address-settings>
<address-setting match="#">
<dead-letter-address>jms.queue.DLQ</dead-letter-address>
<expiry-address>jms.queue.ExpiryQueue</expiry-address>
<redelivery-delay>0</redelivery-delay>
<max-delivery-attempts>1</max-delivery-attempts>
6. Update prbeans/META-INF/ejb-jar.xml file present in prpc_j2ee14_jboss61JBM.ear file and set value of activation config property named "maxSession" to a value that is equal to or lower than the MDB pool size as shown in below example:
<message-driven>
<ejb-name>PRAsync</ejb-name>
<ejb-class>com.pega.pegarules.internal.etier.mdb.PRAsyncBoot</ejb-class>
<messaging-type>javax.jms.MessageListener</messaging-type>
<transaction-type>Bean</transaction-type>
<activation-config>
--------
<activation-config-property>
<activation-config-property-name>maxSession</activation-config-property-name>
<activation-config-property-value>100</activation-config-property-value>
</activation-config-property>
</activation-config>
</message-driven>
<ejb-name>PRAsync</ejb-name>
<ejb-class>com.pega.pegarules.internal.etier.mdb.PRAsyncBoot</ejb-class>
<messaging-type>javax.jms.MessageListener</messaging-type>
<transaction-type>Bean</transaction-type>
<activation-config>
--------
<activation-config-property>
<activation-config-property-name>maxSession</activation-config-property-name>
<activation-config-property-value>100</activation-config-property-value>
</activation-config-property>
</activation-config>
</message-driven>
Published May 2, 2017 - Updated October 8, 2020
Have a question? Get answers now.
Visit the Collaboration Center to ask questions, engage in discussions, share ideas, and help others.