Support Article
Nodes do not join cluster post update to Pega 8.1.1
Summary
After upgrading to Pega Platform 8.1.1, the nodes do not join the Hazelcast cluster.
Error Messages
[7e5.cached.thread-10] [STANDARD] [ ] [ ] (bs.AbstractSystemOperationsJob) ERROR - Error during job execution: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.pega.pegarules.management.internal.system.operations.jobs.AbstractSystemOperationsJob$SystemOperationsJobRunnable.invokeMethodOnClass(AbstractSystemOperationsJob.java:114)
at com.pega.pegarules.management.internal.system.operations.jobs.AbstractSystemOperationsJob$SystemOperationsJobRunnable.run(AbstractSystemOperationsJob.java:107)
at com.pega.pegarules.session.internal.PRSessionProviderImpl.performTargetActionWithLock(PRSessionProviderImpl.java:1366)
at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:1109)
at com.pega.pegarules.session.internal.PRSessionProviderImpl.doWithRequestorLocked(PRSessionProviderImpl.java:916)
at com.pega.pegarules.management.internal.system.operations.jobs.AbstractSystemOperationsJob.invoke(AbstractSystemOperationsJob.java:89)
at com.pega.pegarules.management.internal.system.operations.jobs.AbstractSystemOperationsJob.execute(AbstractSystemOperationsJob.java:58)
at com.pega.pegarules.management.internal.system.operations.jobs.AbstractSystemOperationsJob.execute(AbstractSystemOperationsJob.java:38)
at com.pega.pegarules.cluster.internal.PRPCTask.call(PRPCTask.java:130)
at com.pega.pegarules.cluster.internal.PRPCTask.call(PRPCTask.java:53)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
.
.
.
com.pega.pegarules.management.internal.system.operations.jobs.ClusterManagementJob.getCurrentNodeTypes(ClusterManagementJob.java:132)
at com.pega.pegarules.management.internal.system.operations.jobs.ClusterManagementJob.getConcatenatedNodetypesString(ClusterManagementJob.java:97)
at com.pega.pegarules.management.internal.system.operations.jobs.ClusterManagementJob.getHighLevelNodeInfoAsJSON(ClusterManagementJob.java:126)
at com.pega.pegarules.management.internal.system.operations.jobs.ClusterManagementJob.getHighLevelNodeInfo(ClusterManagementJob.java:155)
at com.pega.pegarules.management.internal.system.operations.jobs.ClusterManagementJob.invokeOperation(ClusterManagementJob.java:316)
... 22 more
Steps to Reproduce
Upgrade Pega Platform 7.4 to Pega Platform 8.1.1.
Root Cause
A defect in Pegasystems’ code or rules.
Every node must discover the cluster topology on startup. In the current implementation, remote execution is used to communicate between nodes in the cluster.
On node startup, the following occurred:
- A node requests information from all the currently running nodes in the cluster (Hazelcast call to all nodes).
- Every other node will know that a new node joined and will try to build a new topology picture by enquiring all nodes about their status
- At this point, all the nodes are trying to communicate with each other and are waiting for a response.
Resolution
Perform the following local-change:
Add the below Dynamic System Settings (DSS),
- Owning Ruleset: Pega-Engine
- Setting Purpose: prconfig/systemoperations/job/timeoutinsecond/default
- Value: 120
The recommended change is to install the current Pega Platform 8.1 patch release.
Alternatively, update to Pega Platform 8.1.4.
Published August 15, 2019 - Updated December 2, 2021
Have a question? Get answers now.
Visit the Collaboration Center to ask questions, engage in discussions, share ideas, and help others.