Skip to main content

This content has been archived and is no longer being updated. Links may not function; however, this content may be relevant to outdated versions of the product.

Support Article

Agents experiencing OutOfMemoryError: Failed to create a thread

SA-12061

Summary



Application complaining of not able to create threads and thus experiencing an “OutOfMemoryError: Failed to create a thread” and requiring restart of application.

Error Messages



[7/7/15 0:39:46:679 EDT] 0000009f SystemErr R java.lang.OutOfMemoryError: Failed to create a thread: retVal -1073741830, errno 11
at java.lang.Error.<init>(Error.java:82)
at java.lang.VirtualMachineError.<init>(VirtualMachineError.java:64)
at java.lang.OutOfMemoryError.<init>(OutOfMemoryError.java:69)
at java.lang.Thread.startImpl(Native Method)
at java.lang.Thread.start(Thread.java:958)
at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:981)
at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1404)
at com.hazelcast.spi.impl.BasicOperationScheduler.execute(BasicOperationScheduler.java:148)
at com.hazelcast.spi.impl.BasicOperationService.receive(BasicOperationService.java:206)
at com.hazelcast.spi.impl.NodeEngineImpl.handlePacket(NodeEngineImpl.java:239)
at com.hazelcast.nio.NodeIOService.handleMemberPacket(NodeIOService.java:99)
at com.hazelcast.nio.SocketPacketReader.enqueueFullPacket(SocketPacketReader.java:54)
at com.hazelcast.nio.SocketPacketReader.access$200(SocketPacketReader.java:26)
at com.hazelcast.nio.SocketPacketReader$DefaultPacketReader.readPacket(SocketPacketReader.java:69)
at com.hazelcast.nio.SocketPacketReader.read(SocketPacketReader.java:49)
at com.hazelcast.nio.ReadHandler.handle(ReadHandler.java:72)
at com.hazelcast.nio.InSelectorImpl.handleSelectionKey(InSelectorImpl.java:33)
at com.hazelcast.nio.AbstractIOSelector.run(AbstractIOSelector.java:126)

Steps to Reproduce



Setup PRPC on Linux server running multiple applications.

Root Cause



This issue is due to less number of user processes configured at OS level.
For most cases of older versions of Linux this value will be defaulted to around 2048. For out of the box Red Hat Enterprise Linux (RHEL) 6 the default value for nproc will be set to 1024. This low default setting for larger systems will not allow for enough threads in all processes.

Resolution



Increase NPROC to higher than default value in Operating system.

 
When using WebSphere Application Server, ulimits can be set to fix or tune around a number of problems. For more on how to set a ulimit refer to the "Guidelines for setting ulimits" Technote which goes into detail on setting different ulimits on various operating systems and the difference between the soft and hard limit. This article is concerned particularly with the "-u" ulimit or "nproc" limit which affects the number of processes allowed by a single user running WebSphere Application Server.

The nproc limit usually only counts processes on a server towards determining this number. Linux systems running WebSphere Application Server are a particular case. The nproc limit on Linux counts the number of threads within all processes that can exist for a given user. To determine the ulimit settings of a WebSphere Application Server process running on Linux refer to "How to determine the ulimit settings of a running WebSphere Application Server process on Linux".

 
Suggest Edit

Published July 13, 2015 - Updated October 8, 2020

Did you find this content helpful? Yes No

Have a question? Get answers now.

Visit the Collaboration Center to ask questions, engage in discussions, share ideas, and help others.

We'd prefer it if you saw us at our best.

Pega Community has detected you are using a browser which may prevent you from experiencing the site as intended. To improve your experience, please update your browser.

Close Deprecation Notice
Contact us