Support Article
Production system down and cannot access login screen
SA-10150
Summary
Production system is down. Cannot access login screen. Getting alerts from server that it is up, then down, repeatedly.
Error Messages
com.pega.pegarules.pub.context.RequestorLockException: Unable to synchronize on requestor H374C72346C1A09854E0376C4174AFD6F within 120 seconds: (thisThread = WebContainer : 14) (originally locked by = WebContainer : 15) (finally locked by = WebContainer : 15)
Thread[WebContainer : 15,5,main]
java.util.Collections$SynchronizedCollection.size(Collections.java:476)
at java.util.AbstractSet.equals(AbstractSet.java:57)
at java.util.Collections$SynchronizedSet.equals(Collections.java:829)
at com.ibm.ws.rsadapter.spi.WSManagedConnectionFactoryImpl$Equals.checkPrivateCredentials(WSManagedConnectionFactoryImpl.java:3334)
at com.ibm.ws.rsadapter.spi.WSManagedConnectionFactoryImpl$Equals.run(WSManagedConnectionFactoryImpl.java:3243)
at com.ibm.ws.security.util.AccessController.doPrivileged(AccessController.java:63)
at com.ibm.ws.rsadapter.spi.WSManagedConnectionFactoryImpl.matchManagedConnections(WSManagedConnectionFactoryImpl.java:2272)
at com.ibm.ejs.j2c.FreePool.getMCWrapperFromMatch(FreePool.java:1894)
at com.ibm.ejs.j2c.FreePool.getFreeConnection(FreePool.java:883)
at com.ibm.ejs.j2c.PoolManager.reserve(PoolManager.java:2218)
at com.ibm.ejs.j2c.ConnectionManager.allocateMCWrapper(ConnectionManager.java:1063)
at com.ibm.ejs.j2c.ConnectionManager.allocateConnection(ConnectionManager.java:700)
at com.ibm.ws.rsadapter.jdbc.WSJdbcDataSource.getConnection(WSJdbcDataSource.java:668)
at com.ibm.ws.rsadapter.jdbc.WSJdbcDataSource.getConnection(WSJdbcDataSource.java:635)
Unable to synchronize on requestor H03C47E647124E864BAB92A98CBB7BF74 within 120 seconds: (thisThread = WebContainer : 28) (originally locked by = WebContainer : 2) (finally locked by = WebContainer : 2)
Thread[WebContainer : 2,5,main]
java.util.Collections$SynchronizedSet.hashCode(Collections.java:835)
at com.ibm.ejs.j2c.PoolManager$SubjectHashCode.run(PoolManager.java:4957)
at com.ibm.ws.security.util.AccessController.doPrivileged(AccessController.java:63)
at com.ibm.ejs.j2c.PoolManager.computeHashCode(PoolManager.java:4632)
at com.ibm.ejs.j2c.PoolManager.reserve(PoolManager.java:2199)
at com.ibm.ejs.j2c.ConnectionManager.allocateMCWrapper(ConnectionManager.java:1063)
at com.ibm.ejs.j2c.ConnectionManager.allocateConnection(ConnectionManager.java:700)
at com.ibm.ws.rsadapter.jdbc.WSJdbcDataSource.getConnection(WSJdbcDataSource.java:668)
at com.ibm.ws.rsadapter.jdbc.WSJdbcDataSource.getConnection(WSJdbcDataSource.java:635)
at com.pega.pegarules.data.internal.access.DataSourceManager.getConnection(DataSourceManager.java:318)
at com.pega.pegarules.data.internal.access.DatabaseConnectionImpl.activate(DatabaseConnectionImpl.java:1285)
at com.pega.pegarules.data.internal.access.ConnectionManagerImpl.getConnection(ConnectionManagerImpl.java:1230)
at com.pega.pegarules.data.internal.access.ConnectionManagerImpl.getConnection(ConnectionManagerImpl.java:947)
Steps to Reproduce
There is no specific use case to reproduce this issue.
Root Cause
The root cause of this problem is in a third-party product defect. The root cause is that the application fails to acquire connections due to deadlock threads (as below) hanging the entire JVM.
Resolution
This is a known issue with the IBM websphere data source and the solution is to create a Data source custom property "useRRASetEquals" of type java.lang.String and value set to “true” for the PRPC datasource.
Published June 12, 2015 - Updated October 8, 2020
Have a question? Get answers now.
Visit the Collaboration Center to ask questions, engage in discussions, share ideas, and help others.