Support Article
Multiple Hazelcast exceptions occur in the logs
SA-49207
Summary
Multiple Hazelcast exceptions occur in the logs.
Error Messages
[erClockSynchDaemon-0] [ STANDARD] [ ] (nternal.PRClusterHazelcastImpl) ERROR - java.util.concurrent.ExecutionException: com.hazelcast.core.OperationTimeoutException: No response for 120000 ms. Aborting invocation! BasicInvocationFuture{invocation=BasicInvocation{ serviceName='hz:impl:executorService', op=Operation{serviceName='hz:impl:executorService', callId=5552714, invocationTime=1512361550565, waitTimeout=-1, callTimeout=60000}, partitionId=-1, replicaIndex=0, tryCount=250, tryPauseMillis=500, invokeCount=1, callTimeout=60000, backupsExpected=0, backupsCompleted=0}, response=null, done=false} No response has been received! backups-expected:0 backups-completed: 0
[201c4eb202a.response] [ STANDARD] [ ] (hazelcast.spi.OperationService) FATAL - [93bc5cd63e21fba5ebdfe201c4eb202a] [3.4.1] While processing response...
com.hazelcast.core.HazelcastException: No invocation for response: com.hazelcast.spi.impl.CallTimeoutResponse@411e85c8
at com.hazelcast.spi.impl.BasicOperationService$ResponsePacketHandler.notifyRemoteCall(BasicOperationService.java:602)
at com.hazelcast.spi.impl.BasicOperationService$ResponsePacketHandler.handle(BasicOperationService.java:586)
at com.hazelcast.spi.impl.BasicOperationService$ResponsePacketHandler.access$1300(BasicOperationService.java:579)
at com.hazelcast.spi.impl.BasicOperationService$BasicDispatcherImpl.dispatch(BasicOperationService.java:564)
at com.hazelcast.spi.impl.BasicOperationScheduler$ResponseThread.process(BasicOperationScheduler.java:532)
at com.hazelcast.spi.impl.BasicOperationScheduler$ResponseThread.doRun(BasicOperationScheduler.java:524)
at com.hazelcast.spi.impl.BasicOperationScheduler$ResponseThread.run(BasicOperationScheduler.java:501)
[erClockSynchDaemon-0] [ STANDARD] [ ] (nternal.PRClusterHazelcastImpl) ERROR - This node's clock has drifted beyond configured threshold of 10 sec, and has the oldest time in the cluster; the actual delta is 56.79 seconds.Current Member Clocks - be sure to check them all
Steps to Reproduce
Unknown.
Root Cause
A defect or configuration issue in the operating environment. There was a clock drift of around 2 minutes between the application tier and the utility tier.
Resolution
Perform the following local-change:
- Check the version of Hazelcast in the system using following query:
SELECT * FROM pr_engineclasses WHERE pzjar like '%hazelcast%'; - Re-sync both the server clocks using National Institute of Standards and Technology (NIST) as Network Time Protocol (NTP)
Published March 12, 2018 - Updated October 8, 2020
Have a question? Get answers now.
Visit the Collaboration Center to ask questions, engage in discussions, share ideas, and help others.