Users are intermittantly unable to login to PRPC. Existing users are able to work, but report slow performance. Examination of the Pega logfile shows java.lang.OutOfMemoryError errors. Note that the environment is a 32bit JVM/WebSphere implementation.
Multiple errror conditions were observed. The following is one of several out-of-memory errors observed in the logfile:
2014-07-17 05:03:19,482 [j2ee14_ws,maxpri=10]] [ STANDARD] [ ] ( internal.async.AgentQueue) WARN - Out of memory exception caught in agent queue: OrgXXHFW #6: Org-FW-YYY-Work.RetrySLAProcess
java.lang.OutOfMemoryError: ZIP002:OutOfMemoryError, MEM_ERROR in deflate_init2
at java.util.zip.Deflater.init(Native Method)
Steps to Reproduce
To reproduce, restart Process Commander, and monitor the PRPC log. Intermittantly, this error (or a derivative) is observed. No specific reproduction procedure was found to trigger this error condition. Also note, the above error condition is one of several similar out of memory errors found in the logfile.
The root cause was found to be a known defect in the IBM J9 JVM. This defect is discussed here:
Native OutOfMemory problems are a known problem with certain versions of the IBM J9 JVM for AIX and other Unix/Linux
operating systems, and IBM has provided several important updates to correct this issue. SR11 (Service Release 11) of the IBM J9 JVM includes a critical fix for classloader static native memory allocations, and this key importance to your system.
Per the description above, the solution to this SR is to obtain an updated IBM J9 JVM update kit for WebSphere, and apply the update.