Support Article

Data flow is crashing WebLogic application server

SA-29059

Summary



Data flow is crashing WebLogic application server because of Java heap space error.

The user is testing custom data flow to read and merge data from 5 or 6 tables and load them into staging table. When they load 600 to 1 million records, their system is crashing because of Java heap space error.


Error Messages



016-10-05 17:16:51,989 [[EDM:Administrators]] [ STANDARD] [ ] [ EDM:01.01.01] (tor$QueueBasedDataFlowExecutor) ERROR anAddress|anotherAddress anUser- Unexpected error occurred during event processing: com.pega.dsm.dnode.api.dataflow.DataFlowExecutor$QueueBasedDataFlowExecutor$5@7daad6e1
java.lang.OutOfMemoryError: Java heap space


Steps to Reproduce



We are testing custom data flow to read and merge data from 5 or 6 tables and load them into staging table. When we load 600 to 1 million records, our system is crashing because of Java heap space error. Attached the log file for your review.


Root Cause



Duplicate LinkedList with same size were observed in the heap while running data flow referenced by com.pega.dsm.dnode.api.dataflow.DataFlowExecutor$QueueBasedDataFlowExecutor and com.pega.dsm.dnode.impl.prpc.PrpcThreadFactory$PrpcThread.

Resolution



Apply HFix-29913.

 

Published October 12, 2016 - Updated October 23, 2016

Have a question? Get answers now.

Visit the Collaboration Center to ask questions, engage in discussions, share ideas, and help others.