Support Article

Node crashed with OutOfMemory error

SA-31148

Summary



One node of total 9 nodes crashed with an out of memory error and server needed to be restarted to fix the issue.


Error Messages



Log File Monitor - Java OutOfMemoryError – PermGen Space Availability:Solutions Center


Steps to Reproduce



This is an issue observed in runtime, and exact steps are unknown. 


Root Cause



Developers recently implemented a change to the dynamic system setting (DSS) to control prconfig/timeout/browser/default, increasing the value to 12000 (which means that idle requestors will remain in heap for 200 minutes before being passivated and removed).  With the requestor volume (and requestor sizes which can reach up to 90MB), the larger collection consumed enough memory to get very near to max heap (8GB) and to trigger full gcs which slowed down all running threads. 

Resolution



System Administrators have updated the DSS back to 1200 seconds (20 minutes), and are performing a rolling restart of servers to complete this change.

Published December 6, 2016 - Updated December 9, 2016

Have a question? Get answers now.

Visit the Collaboration Center to ask questions, engage in discussions, share ideas, and help others.