Support Article
Data flow is crashing WebLogic application server
SA-29059
Summary
Data flow is crashing WebLogic application server because of Java heap space error.
The user is testing custom data flow to read and merge data from 5 or 6 tables and load them into staging table. When they load 600 to 1 million records, their system is crashing because of Java heap space error.
Error Messages
016-10-05 17:16:51,989 [[EDM:Administrators]] [ STANDARD] [ ] [ EDM:01.01.01] (tor$QueueBasedDataFlowExecutor) ERROR anAddress|anotherAddress anUser- Unexpected error occurred during event processing: com.pega.dsm.dnode.api.dataflow.DataFlowExecutor$QueueBasedDataFlowExecutor$5@7daad6e1
java.lang.OutOfMemoryError: Java heap space
Steps to Reproduce
We are testing custom data flow to read and merge data from 5 or 6 tables and load them into staging table. When we load 600 to 1 million records, our system is crashing because of Java heap space error. Attached the log file for your review.
Root Cause
Duplicate LinkedList with same size were observed in the heap while running data flow referenced by com.pega.dsm.dnode.api.dataflow.DataFlowExecutor$QueueBasedDataFlowExecutor and com.pega.dsm.dnode.impl.prpc.PrpcThreadFactory$PrpcThread.
Resolution
Apply HFix-29913.
Published October 24, 2016 - Updated December 2, 2021
Have a question? Get answers now.
Visit the Collaboration Center to ask questions, engage in discussions, share ideas, and help others.