Skip to main content

This content has been archived and is no longer being updated. Links may not function; however, this content may be relevant to outdated versions of the product.

Support Article

System hung or crashes with nativce OOM



After few hours of start-up and with no users logged in, the JVM crashes with native OOM and hangs.

Error Messages

Detail "java/lang/OutOfMemoryError" "Failed to create a thread: retVal -1073741830, errno 11" received

Steps to Reproduce

Not Applicable

Root Cause

Java core of thread dump shows thousands of threads in parked state with thread call stack below. This indicates that there are entries in PR_SYS_STATUSNODES that cannot be reached by the JVM (part of auto discovery process within Elastic search)

 "PegaRULES-Search[[unicast_connect]][T#4]" J9VMThread:0x0000000008DD6B00, j9thread_t:0x00007F421087D240, java/lang/Thread:0x000000078A72A978, state:P, prio=5
        (java/lang/Thread getId:0xEC4, isDaemon:true)
        (native thread ID:0x3552, native priority:0x5, native policy:UNKNOWN, vmstate:P, vm thread flags:0x00020001)
        (native stack address range from:0x00007F420D8A5000, to:0x00007F420D8E6000, size:0x41000)
       CPU usage total: 0.000106099 secs
 Parked on: java/util/concurrent/locks/ReentrantLock$NonfairSync@0x00000005061186D0 Owned by: "PegaRULES-Search[[unicast_connect]][T#2]" (J9VMThread:0x000000000554E800, java/lang/Thread:0x00000007860E16A0)
       Heap bytes allocated since last GC cycle=0 (0x0)
       Java callstack:
           at sun/misc/Unsafe.park(Native Method)
           at java/util/concurrent/locks/LockSupport.park( Code))
           at java/util/concurrent/locks/AbstractQueuedSynchronizer.parkAndCheckInterrupt( Code))
           at java/util/concurrent/locks/AbstractQueuedSynchronizer.acquireQueued( Code))
           at java/util/concurrent/locks/AbstractQueuedSynchronizer.acquire( Code))
           at java/util/concurrent/locks/ReentrantLock$NonfairSync.lock( Code))
           at java/util/concurrent/locks/ReentrantLock.lock( Code))
           at com/pega/elasticsearch/common/util/concurrent/KeyedLock.acquire( Code))
           at com/pega/elasticsearch/transport/netty/NettyTransport.connectToNode( Code))
           at com/pega/elasticsearch/transport/netty/NettyTransport.connectToNodeLight( Code))
           at com/pega/elasticsearch/transport/TransportService.connectToNodeLight( Code))
           at com/pega/elasticsearch/discovery/zen/ping/unicast/UnicastZenPing$ Code))
           at java/util/concurrent/ThreadPoolExecutor.runWorker( Code))
           at java/util/concurrent/ThreadPoolExecutor$ Code))
           at java/lang/ Code))
       No native callstack available for this thread


Perform the following local-change:

- Recycle all JVM’s
- This time when you login, you should see ONLY one Index host configuration which comes from Dynamic system setting  indexing/hostid and indexing/explicitindexdir
- Change the index host settings if required i.e if the DSS values above are incorrect and submit.
- Please note that at runtime, search settings are read from PR_SYS_STATUSNODES table

Published March 16, 2016 - Updated October 8, 2020

Was this useful?

0% found this useful

Have a question? Get answers now.

Visit the Collaboration Center to ask questions, engage in discussions, share ideas, and help others.

Did you find this content helpful?

Want to help us improve this content?

We'd prefer it if you saw us at our best.

Pega Community has detected you are using a browser which may prevent you from experiencing the site as intended. To improve your experience, please update your browser.

Close Deprecation Notice
Contact us