Support Article

Agents last runtime are not getting updated in SMA

SA-36764

Summary



Agents last run-time are not getting updated in SMA.

Error Messages



1. 2017-04-13 22:31:25,261 [ PegaRULES-Batch-2] [ STANDARD] [ PegaRULES:06.03] ( async.agent.QueueManagerImpl) ERROR - Problem with browse of System-Work-Indexer
com.pega.pegarules.pub.database.DatabaseException: There was a problem getting a list: code: 1423 SQLState: 72000 Message: ORA-01423: error encountered while checking for extra rows in exact fetch
ORA-01089: immediate shutdown in progress - no operations are permitted
ORA-06512: at "RULES.SPPR_SYS_RESERVEQUEUEITEM_B", line 110
ORA-06512: at line 1

2. 2017-04-13 22:31:26,829 [ PegaRULES-Batch-2] [ STANDARD] [ PegaRULES:06.03] ( internal.access.Opener) ERROR - There was a problem opening a database instance YOUR_WORK_CLASS T-548527
com.pega.pegarules.pub.database.ConnectionException: Database-General There was a problem opening a database instance YOUR_WORK_CLASS T-548527 1089 08006 ORA-01089: immediate shutdown in progress - no operations are permitted

From: (B8347EDEE9606EFFDDC50E4668DD9E4ED)
SQL: select pzPVStream from pc_YOUR_WORK_CLASS where pzInsKey = ?
SQL Inserts: <YOUR_WORK_CLASS T-548527>

Caused by SQL Problems.
Problem #1, SQLState 08006, Error code 1089: java.sql.SQLRecoverableException: ORA-01089: immediate shutdown in progress - no operations are permitted

at com.pega.pegarules.data.internal.access.ExceptionInformation.createExceptionDueToDBFailure(ExceptionInformation.java:163)

3. 2017-04-13 13:43:10,008 [ PegaRULES-Batch-2] [ STANDARD] [ IPI:02.01.01] (.YOUR_WORK_CLASS.Action) ERROR - The case WA-220085 could not be updated because Fail: Database-Instance-NotFound pxObjClass = "Data-Admin-Operator-ID", pyUserIdentifier = "Digital"
2017-04-13 13:43:11,711 [ PegaRULES-Batch-2] [ STANDARD] [ IPI:02.01.01] (em_Queue_IPI_CreateCase.Action) ERROR - obj-save failed.
com.pega.pegarules.pub.database.LockGoneException: Save, Delete or Commit has failed because lock "YOUR_WORK_CLASS CS-3562" is not held
From: (B96F3B4A08116F25B69C7A16E7BD5EFEC)
at com.pega.pegarules.data.internal.access.Saver.checkRequiredLock(Saver.java:1566)
at com.pega.pegarules.data.internal.access.Saver.saveDeferred(Saver.java:352)
at com.pega.pegarules.data.internal.access.DatabaseImpl.saveDeferred(DatabaseImpl.java:4996)
at com.pegarules.generated.activity.ra_action_generateinitialprojectplan_386b278f147175d23d6eeffcb274bead.step6_circum0(ra_action_generateinitialprojectplan_386b278f147175d23d6eeffcb274bead.java:1313)
at com.pegarules.generated.activity.ra_action_generateinitialprojectplan_386b278f147175d23d6eeffcb274bead.perform(ra_action_generateinitialprojectplan_386b278f147175d23d6eeffcb274bead.java:186)


Steps to Reproduce



Launch the Agent page on SMA and observe that the Last Finish Date is not getting populated.


Root Cause



A defect or configuration issue in the operating environment: After looking into the PRPC logs we observed that the oracle shutdown got triggered abruptly, this made us to look into the Oracle logs and found that the below error got thrown on the Oracle side:

Errors in file /smartpaas-db/oracle/app/diag/rdbms/prpc/PRPC/trace/PRPC_ora_18291.trc (incident=86692):
ORA-07445: exception encountered: core dump [kgghash()+367] [SIGSEGV] [ADDR:0x2B203B36D000] [PC:0x70B322F] [Address not mapped to object] []


This can get thrown for various reasons, in this case we found it to be memory tuning issue. After comparing it with the the working environment, we observed that in working environment memory allocated to Oracle is 12gb and in this non-working environment it is 4GB.

Values which are of concern are:

Memory_Max_Target 4G
Memory_Target 3G

Resolution



Make the following change to the operating environment:

1. Get the memory tuning done in the DB node.
2. For Pega (On premises instances) suggested values for Oracle are:


Memory_Max_Target 12G
Memory_Target 10G
3. Restart Oracle instance and App server.

Published April 19, 2017 - Updated April 21, 2017

Have a question? Get answers now.

Visit the Collaboration Center to ask questions, engage in discussions, share ideas, and help others.