Support Article
Production Load Balancer down and one node down with exceptions
SA-31503
Summary
There are hazelcast communication error messages in the logs for every second and logs are growing at rapid phase. One of the node in production is down.
Error Messages
Error 503: PegaRULES server not available while launching the URL.
Following is the repetitive error from Node 2 sysout.log which is happening every few milliseconds:
[12/15/16 16:37:34:911 CET] 00000098 InternalParti W com.hazelcast.partition.InternalPartitionService [node2]:5701 [6971d459d3c43fe825c043f7e6116d77] [3.4.1] Following unknown addresses are found in partition table sent from master[Address[node1]:5701]. (Probably they have recently joined or left the cluster.) {
Address[DR Node1]:5701
Address[DR Node 2]:5701
}
Steps to Reproduce
Not applicable
Root Cause
A defect or configuration issue in the operating environment.
There was no communication between production nodes and DR nodes. DR nodes are not used in this environment.
Resolution
Here’s the explanation for the reported behavior:
There was no communication between production nodes and DR nodes, shutdown the DR nodes to address the hazelcast errors.
Restarting the problematic node resolves the issue.
Published December 21, 2016 - Updated October 8, 2020
Have a question? Get answers now.
Visit the Collaboration Center to ask questions, engage in discussions, share ideas, and help others.