Skip to main content

This content has been archived and is no longer being updated. Links may not function; however, this content may be relevant to outdated versions of the product.

Support Article

ORA-02391and NoClassDefFoundErrors in Marketing REST services

SA-39639

Summary



The following errors occur in logs while executing a strategy inside a Data Flow in a headless Decisioning application.

The issue is occurring after some minute from application start. Restarting the application is not solving the issue.

Error Messages

( internal.access.DatabaseImpl) ERROR localhost|127.0.0.1 - There was a problem with the database when getting a list:
com.pega.pegarules.pub.database.DatabaseException: Database-General Problem encountered when getting connection for database pegarules 604 60000 ORA-00604: error occurred at recursive SQL level 1
ORA-06502: PL/SQL: numeric or value error
ORA-06512: at line 148
ORA-02391: exceeded simultaneous SESSIONS_PER_USER limit
...
DSRA0010E: SQL State = 60000, Error Code = 604
at com.pega.pegarules.data.internal.access.ExceptionInformation.__AW_createExceptionDueToDBFailure(ExceptionInformation.java:262)
at com.pega.pegarules.data.internal.access.ExceptionInformation.createExceptionDueToDBFailure(ExceptionInformation.java)
at com.pega.pegarules.data.internal.access.ConnectionManagerImpl.getConnectionFromDataSourceIdentity(ConnectionManagerImpl.java:1087)
at com.pega.pegarules.data.internal.access.ConnectionManagerImpl.getConnection(ConnectionManagerImpl.java:1216)
at com.pega.pegarules.data.internal.access.ConnectionManagerImpl.getConnection(ConnectionManagerImpl.java:1156)
at com.pega.pegarules.data.internal.access.ConnectionManagerImpl.getConnection(ConnectionManagerImpl.java:347)
at com.pega.pegarules.data.internal.access.ThreadConnectionStoreImpl.getConnection(ThreadConnectionStoreImpl.java:426)
at com.pega.pegarules.data.internal.access.RDBPageResultPackager.getConnection(RDBPageResultPackager.java:924)
at com.pega.pegarules.data.internal.access.DatabaseImpl.__AW_list(DatabaseImpl.java:6279)
at com.pega.pegarules.data.internal.access.DatabaseImpl.list(DatabaseImpl.java)
at com.pega.pegarules.data.internal.access.DatabaseImpl.executeRDB(DatabaseImpl.java:6932)
...
Caused by:
java.sql.SQLException: ORA-00604: error occurred at recursive SQL level 1
ORA-06502: PL/SQL: numeric or value error
ORA-06512: at line 148
ORA-02391: exceeded simultaneous SESSIONS_PER_USER limit
DSRA0010E: SQL State = 60000, Error Code = 604
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:439)
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:388)
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:381)
at oracle.jdbc.driver.T4CTTIfun.processError(T4CTTIfun.java:564)
at oracle.jdbc.driver.T4CTTIoauthenticate.processError(T4CTTIoauthenticate.java:431)
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:436)
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:186)
at oracle.jdbc.driver.T4CTTIoauthenticate.doOAUTH(T4CTTIoauthenticate.java:366)
at oracle.jdbc.driver.T4CTTIoauthenticate.doOAUTH(T4CTTIoauthenticate.java:752)
at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:359)
at oracle.jdbc.driver.PhysicalConnection.<init>(PhysicalConnection.java:531)
at oracle.jdbc.driver.T4CConnection.<init>(T4CConnection.java:221)
at oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:32)
at oracle.jdbc.driver.OracleDriver.__AW_connect(OracleDriver.java:503)
at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java)
at oracle.jdbc.pool.OracleDataSource.getPhysicalConnection(OracleDataSource.java:280)
at oracle.jdbc.xa.client.OracleXADataSource.getPooledConnection(OracleXADataSource.java:466)
at oracle.jdbc.xa.client.OracleXADataSource.getXAConnection(OracleXADataSource.java:154)
at oracle.jdbc.xa.client.OracleXADataSource.getXAConnection(OracleXADataSource.java:99)
at com.ibm.ws.rsadapter.spi.InternalGenericDataStoreHelper$1.run(InternalGenericDataStoreHelper.java:1365)
at com.ibm.ws.security.util.AccessController.doPrivileged(AccessController.java:118)
at com.ibm.ws.rsadapter.spi.InternalGenericDataStoreHelper.getPooledConnection(InternalGenericDataStoreHelper.java:1384)
at com.ibm.ws.rsadapter.spi.WSRdbDataSource.getPooledConnection(WSRdbDataSource.java:2154)
at com.ibm.ws.rsadapter.spi.WSManagedConnectionFactoryImpl.getConnection(WSManagedConnectionFactoryImpl.java:1787)
at com.ibm.ws.rsadapter.spi.WSManagedConnectionFactoryImpl.createManagedConnection(WSManagedConnectionFactoryImpl.java:1557)
at com.ibm.ws.rsadapter.spi.WSManagedConnectionFactoryImpl.createManagedConnection(WSManagedConnectionFactoryImpl.java:1113)
at com.ibm.ejs.j2c.FreePool.createManagedConnectionWithMCWrapper(FreePool.java:2161)
at com.ibm.ejs.j2c.FreePool.createOrWaitForConnection(FreePool.java:1839)
at com.ibm.ejs.j2c.PoolManager.reserve(PoolManager.java:3818)
at com.ibm.ejs.j2c.PoolManager.reserve(PoolManager.java:3094)
at com.ibm.ejs.j2c.ConnectionManager.allocateMCWrapper(ConnectionManager.java:1548)
at com.ibm.ejs.j2c.ConnectionManager.allocateConnection(ConnectionManager.java:1031)
at com.ibm.ws.rsadapter.jdbc.WSJdbcDataSource.getConnection(WSJdbcDataSource.java:644)
at com.ibm.ws.rsadapter.jdbc.WSJdbcDataSource.__AW_getConnection(WSJdbcDataSource.java:611)
at com.ibm.ws.rsadapter.jdbc.WSJdbcDataSource.getConnection(WSJdbcDataSource.java)
at com.pega.pegarules.data.internal.access.DataSourceManager.getConnection(DataSourceManager.java:221)
at com.pega.pegarules.data.internal.access.DatabaseConnectionImpl.activate(DatabaseConnectionImpl.java:1483)
at com.pega.pegarules.data.internal.access.ConnectionManagerImpl.getConnectionFromDataSourceIdentity(ConnectionManagerImpl.java:1050)
... 105 more
...

[ YourAPPLICATION:01.01.01] ( internal.services.ServiceAPI) ERROR localhost|127.0.0.1|Rest|YourServicePackage|Services|YourService|A492EAB4C903FA633A1D3BE266D41B101 - Rest service [YourServicePackage][Services][YourService] failed:PRRuntimeError:com.pega.pegarules.pub.PRRuntimeError: PRRuntimeError
at com.pega.pegarules.session.internal.mgmt.base.ThreadRunner.__AW_runActivitiesAlt(ThreadRunner.java:712)
at com.pega.pegarules.session.internal.mgmt.base.ThreadRunner.runActivitiesAlt(ThreadRunner.java)
at com.pega.pegarules.session.internal.mgmt.PRThreadImpl.runActivitiesAlt(PRThreadImpl.java:466)
at com.pega.pegarules.integration.engine.internal.services.ServiceAPI.runActivities(ServiceAPI.java:1951)
at com.pega.pegarules.session.external.engineinterface.service.EngineAPI.__AW_processRequestInner(EngineAPI.java:390)
at com.pega.pegarules.session.external.engineinterface.service.EngineAPI.processRequestInner(EngineAPI.java)
at sun.reflect.GeneratedMethodAccessor117.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:56)
at java.lang.reflect.Method.invoke(Method.java:620)
at com.pega.pegarules.session.internal.PRSessionProviderImpl.performTargetActionWithLock(PRSessionProviderImpl.java:1277)
at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:1015)
at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:848)
at com.pega.pegarules.session.external.engineinterface.service.EngineAPI.__AW_processRequest(EngineAPI.java:331)
at com.pega.pegarules.session.external.engineinterface.service.EngineAPI.processRequest(EngineAPI.java)
at com.pega.pegarules.integration.engine.internal.services.http.HTTPService.__AW_invoke(HTTPService.java:457)
at com.pega.pegarules.integration.engine.internal.services.http.HTTPService.invoke(HTTPService.java)
at com.pega.pegarules.session.internal.engineinterface.etier.impl.EngineImpl._invokeEngine_privact(EngineImpl.java:327)
at com.pega.pegarules.session.internal.engineinterface.etier.impl.EngineImpl.__AW_invokeEngine(EngineImpl.java:270)
at com.pega.pegarules.session.internal.engineinterface.etier.impl.EngineImpl.invokeEngine(EngineImpl.java)
at com.pega.pegarules.session.internal.engineinterface.etier.ejb.EngineBean.__AW_invokeEngine(EngineBean.java:225)
at com.pega.pegarules.session.internal.engineinterface.etier.ejb.EngineBean.invokeEngine(EngineBean.java)
at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:56)
at java.lang.reflect.Method.invoke(Method.java:620)
at com.pega.pegarules.internal.bootstrap.PRBootstrap.invokeMethod(PRBootstrap.java:370)
at com.pega.pegarules.internal.bootstrap.PRBootstrap.invokeMethodPropagatingThrowable(PRBootstrap.java:411)
at com.pega.pegarules.boot.internal.extbridge.AppServerBridgeToPega.invokeMethodPropagatingThrowable(AppServerBridgeToPega.java:224)
at com.pega.pegarules.boot.internal.extbridge.AppServerBridgeToPega.invokeMethodPropagatingException(AppServerBridgeToPega.java:246)
at com.pega.pegarules.internal.etier.ejb.EngineBeanBoot.__AW_invokeEngine(EngineBeanBoot.java:168)
at com.pega.pegarules.internal.etier.ejb.EngineBeanBoot.invokeEngine(EngineBeanBoot.java)
at com.pega.pegarules.internal.etier.interfaces.EJSLocalStatelessEngineBMT_f2439d86.invokeEngine(Unknown Source)
at com.pega.pegarules.priv.context.JNDIEnvironment.invokeEngineInner(JNDIEnvironment.java:278)
at com.pega.pegarules.priv.context.JNDIEnvironment.invokeEngine(JNDIEnvironment.java:223)
at com.pega.pegarules.web.impl.WebStandardImpl.__AW_makeEtierRequest(WebStandardImpl.java:574)
at com.pega.pegarules.web.impl.WebStandardImpl.makeEtierRequest(WebStandardImpl.java)
at com.pega.pegarules.web.impl.WebStandardImpl.__AW_doPost(WebStandardImpl.java:374)
at com.pega.pegarules.web.impl.WebStandardImpl.doPost(WebStandardImpl.java)
...
Caused by: java.lang.NoClassDefFoundError: com/google/common/base/Throwables
at com.pega.dsm.dnode.impl.stream.DataObservableImpl.await(DataObservableImpl.java:122)
at com.pega.dsm.dnode.impl.stream.DataObservableImpl.await(DataObservableImpl.java:87)
at com.pegarules.generated.activity.ra_action_returnnba_360b62294c581f81606a9ae8118c3a9a.__AW_step6_circum0(ra_action_returnnba_360b62294c581f81606a9ae8118c3a9a.java:744)
at com.pegarules.generated.activity.ra_action_returnnba_360b62294c581f81606a9ae8118c3a9a.step6_circum0(ra_action_returnnba_360b62294c581f81606a9ae8118c3a9a.java)
at com.pegarules.generated.activity.ra_action_returnnba_360b62294c581f81606a9ae8118c3a9a.__AW_perform(ra_action_returnnba_360b62294c581f81606a9ae8118c3a9a.java:174)
at com.pegarules.generated.activity.ra_action_returnnba_360b62294c581f81606a9ae8118c3a9a.perform(ra_action_returnnba_360b62294c581f81606a9ae8118c3a9a.java)
at com.pega.pegarules.session.internal.mgmt.Executable.__AW_doActivity(Executable.java:3510)
at com.pega.pegarules.session.internal.mgmt.Executable.doActivity(Executable.java)
at com.pega.pegarules.session.internal.mgmt.base.ThreadRunner.__AW_runActivitiesAlt(ThreadRunner.java:646)
... 66 more
...


Steps to Reproduce

  1. Expose a REST decision service in Load Balancer.
  2. Route more than 300 requests per second to the service.


Root Cause



A defect or configuration issue in the operating environment.

The issue was caused by the inability to create new JDBC connections due to "ORA-02391: exceeded simultaneous SESSIONS_PER_USER limit" errors on the database.
Issue was caused by the following configuration issues:
  • An incorrect configuration of the database connection pool:
    The maximum amount of connections was larger than the SESSIONS_PER_USER limit on the database
  • An unlimited requestor pool used in the Service Package associated with REST decision services.

Requestor pools protect the system and the underlying resources from being overloaded.
A tuned requestor pool applies back pressure when overloaded, slowing down and eventually rejecting requests above a threshold arrival rate.
This threshold should be tuned to the rate of requests which the system is capable of processing.


Resolution



Perform the following local-changes:
  • Set a positive value for Maximum Active Requestors in the Pooling tab of the Service.
  • Reduce the maximum amount of JDBC Connections in the datasource pool to stay within the SESSIONS_PER_USER limit on the database.

Tags:

Pega Platform Pega Platform 7.2

Published August 21, 2017 - Updated October 8, 2020

Was this useful?

0% found this useful

Have a question? Get answers now.

Visit the Collaboration Center to ask questions, engage in discussions, share ideas, and help others.

Visit the Collaboration Center

Did you find this content helpful?

Yes
No

Want to help us improve this content?
Suggest an edit

We'd prefer it if you saw us at our best.

Pega Community has detected you are using a browser which may prevent you from experiencing the site as intended. To improve your experience, please update your browser.

Close Deprecation Notice
Contact us