SA-8522

Summary

User gets HTTP 410 error on screen in production environment intermittently at random points in the application.
User had to start over with the case set up.

Error Messages

webpage no longer exists - HTTP 410

Steps to Reproduce

The error occurs intermittently and at random points in the application..

Root Cause

The root cause of this problem is defect/misconfiguration in the operating environment. 

Reviewing the IHS plugin logfiles there were a lot of these occurances for servers in the PRPC Cluster:

[11/Feb/2015:07:15:04.90721] 000043a0 77cffb80 - ERROR: ws_common: ServerActionfromReadRC: ServerIOTimeout fired. Time out 60. retry count 0. serverIOTimeoutRetry -1, retry YES, rc 10 
[11/Feb/2015:07:15:04.90737] 000043a0 77cffb80 - ERROR: ws_common: websphereHandleRequest: Failed to execute the transaction to '<WebSphere_PRPCNode' on host '<hostname>'; will try another one

This means the IHS Plugin ServerIOTimeout setting for the servers in the cluster is not high enough. Every occurrence in the IHS plugin logfile ends up being a HTTP 410 response.

Resolution

The ServerIOTimeout is set to 60 seconds. This implementation had a process where lots of data is gathered from an external system and in some case this takes over 60 seconds. The ServerIOTimeout is changed to 180 seconds and this significantly reduced the retries done by the IHS server. 

This however did not resolve all the issues, there were reports of 410 errors occurring in production but very sporadic in nature.
With the ServerIOTimeout change, it is easier to see what causes the issue.  The gadget is directly calling Code-Security.Logoff in an AJAX request when unloaded. The 410 errors are being caused by called to pyDeleteDocumentPG activity after logoff is called. 

Instead of calling logoff  a change is provided to simply clear the gadget thread page on unload of the Gadget.
This is custom Javascript calling "RemoveThread" with the gadget thread name as the parameter. This is used in another PRPC IAC application and works well.