Support Article

Declare Page Access Causes stuck thread and passivation failure

SA-36226

Summary



Declare Page Access Causes STUCK thread, high CPU utilization and requestor passivation failure.

Users' search indexing node is experiencing
Data Page load exceptions and the Passivation Daemon is failing.

The requestor count go through the roof, over 100, for a jms connection pool that has a maximum of 10 and CPU utilization is high.


Error Messages



GC-Time: 0: 7m #GC-Runs: 10652 #TotalLoadedClasses: 42694
CPU: 16.05% GC: 0.00% HEAP:1298m /3020m NONHEAP: 399m / 560m

TID NAME STATE CPU TOTALCPU BLOCKEDBY
141 [STUCK] ExecuteThread: '48' fo RUNNABLE 98.42% 42.53%
90 [STUCK] ExecuteThread: '16' fo RUNNABLE 98.24% 42.53%
252768 RMI TCP Connection(2)-10.74.19 RUNNABLE 14.95% 0.00%
308 [ACTIVE] ExecuteThread: '80' f WAITING 12.39% 0.08%
146 [ACTIVE] ExecuteThread: '53' f WAITING 11.28% 0.09%
302 [ACTIVE] ExecuteThread: '74' f WAITING 9.46% 0.17%
106 [ACTIVE] ExecuteThread: '26' f WAITING 0.59% 0.11%
252783 JMX server connection timeout TIMED_WAITING 0.36% 0.00%
33 ExecuteThread: '0' for queue: RUNNABLE 0.23% 0.05%
74 weblogic.cluster.MessageReceiv WAITING 0.21% 0.07%
Note: Only top 10 threads (according cpu load) are shown!

"[STUCK] ExecuteThread: '48' for queue: 'weblogic.kernel.Default (self-tuning)'" daemon prio=10 tid=0x00002aba8702f000 nid=0x33b7 runnable [0x00002aba91a16000]
java.lang.Thread.State: RUNNABLE
at java.util.HashMap.put(HashMap.java:498)
at com.pega.pegarules.data.internal.clipboard.ClipboardPageImpl.put(ClipboardPageImpl.java:2192)
at com.pega.pegarules.data.internal.clipboard.ClipboardPageImpl.put(ClipboardPageImpl.java:2015)
at com.pega.pegarules.data.internal.clipboard.ClipboardPageImpl.put(ClipboardPageImpl.java:2005)
at com.pega.pegarules.data.internal.clipboard.ClipboardPageBase.put(ClipboardPageBase.java:1155)
at com.pega.pegarules.data.internal.clipboard.DirectStreamContainer.expandToProperty(DirectStreamContainer.java:544)
at com.pega.pegarules.data.internal.clipboard.DirectStreamContainer.expandAllToProperties(DirectStreamContainer.java:406)
at com.pega.pegarules.data.internal.clipboard.ClipboardPageImpl.expandAll(ClipboardPageImpl.java:1171)
at com.pega.pegarules.session.internal.mgmt.base.handler.ReadOnlyDataPageHandler.setMiscellaneousProperties(ReadOnlyDataPageHandler.java:316)
at com.pega.pegarules.session.internal.mgmt.base.handler.ReadOnlyDataPageHandler.loadDataPage(ReadOnlyDataPageHandler.java:262)
at com.pega.pegarules.session.internal.mgmt.base.handler.ReadOnlyDataPageHandler.findDataPage(ReadOnlyDataPageHandler.java:497)
at com.pega.pegarules.session.internal.mgmt.base.AbstractPageDirectory.findDataPageInCurrentDir(AbstractPageDirectory.java:920)
at com.pega.pegarules.session.internal.mgmt.base.AbstractPageDirectory.getDirectPage(AbstractPageDirectory.java:713)
at com.pega.pegarules.session.internal.mgmt.base.ThreadPageDir.getDirectPage(ThreadPageDir.java:476)
at com.pega.pegarules.session.internal.mgmt.base.ThreadPageDir.getPage(ThreadPageDir.java:434)
at com.pega.pegarules.session.internal.mgmt.PRThreadImpl.getPage(PRThreadImpl.java:539)
at com.pega.pegarules.session.internal.mgmt.Executable.findPage(Executable.java:2081)
at com.pega.pegarules.session.internal.mgmt.Executable.findPage(Executable.java:2034)
at com.pega.pegarules.session.internal.mgmt.Executable.findPage(Executable.java:2016)
at com.pega.pegarules.session.internal.mgmt.authentication.Authentication.getSecuritySettingDeclarePage(Authentication.java:652)
at com.pega.pegarules.session.internal.mgmt.authentication.Authentication.getSecuritySettingAsBoolean(Authentication.java:661)
at com.pega.pegarules.session.internal.mgmt.authentication.Authentication.doAuthentication(Authentication.java:441)
at com.pega.pegarules.integration.engine.internal.services.ServiceAPI.handleAuthentication(ServiceAPI.java:1577)
at com.pega.pegarules.session.external.engineinterface.service.EngineAPI.activityExecutionProlog(EngineAPI.java:548)
at com.pega.pegarules.session.external.engineinterface.service.EngineAPI.processRequestInner(EngineAPI.java:383)
at sun.reflect.GeneratedMethodAccessor349.invoke(Unknown Source)


Steps to Reproduce



None. Unable to reproduce in user environment.


Root Cause



A defect in Pegasystems’ code or rules.

The datapage load may go into an infinite loop when doing expandAll and when the engine tries to load the same data page through an autopopulate or some other call concurrently.

Resolution

Apply HFix-33457.

Published April 7, 2017 - Updated April 21, 2017

Have a question? Get answers now.

Visit the Collaboration Center to ask questions, engage in discussions, share ideas, and help others.