Support Article
High number of Hazelcast threads
SA-31492
Summary
While trying to deploy HFix-29800 and HFix-29157, it was noticed that the number of Hazelcast threads was still sometimes higher than expected in Systems Management Application (SMA).
Error Messages
No error messages, but a high number of threads in SMA.
Steps to Reproduce
Deploy HFix-29800 and HFix-29157 on Websphere in a SPARC/Solaris environment with many CPUs.
Root Cause
In a SunOS environment the nubmer of Hazelcast threads is driven by the number of CPUs on the system might be one of the reason for the issue.
The fixes and settings were provided to try and minimize the number of Hazelcast threads where possible, and to understand where/when one should see intermittent threads started.
High count of the following thread types is the cause for the issue:
- hz._hzInstance_1_2ef39c10182e014af92c2e6c9fbe41df.async.thread-1
- hz._hzInstance_2_7da3a0ca56cf4f6b283566181095e689.generic-operation.thread-0
- hz._hzInstance_2_7da3a0ca56cf4f6b283566181095e689.partition-operation.thread-0
Resolution
Environment details:
Tests were performed on a 16 core Intel based system running SunOS, Tomcat based instances.
Testing utilized 4 to 40 JVMs to simulate a large mulit-core, multi-node environment.
Apply the following hotfixes to resolve the issue:
- HFix-29800
- HFix-29875
- HFix-30617
After applying hotfixes user is able to effectively control each of the three thread types and eliminate async threads due to Visual Business Director (VBD) processing entirely.
Note: Hazelcast async threads do appear at system startup as part of normal processing, however these eventually clear away. Timings on the specific environments need to be validated, but one can expect to see async threads beyond 10 minutes post startup.
Published December 11, 2017 - Updated October 8, 2020
Have a question? Get answers now.
Visit the Collaboration Center to ask questions, engage in discussions, share ideas, and help others.