Support Article
User cannot find a new case on production - Work indexing fails
Summary
Work index file size displays as a negative (for example, -0.01) and user is unable to search for any case.
The user updated the number of nodes from four to 12 and set one node as the indexing node.
This issue occurs despite re-indexing the work or data class manually by changing the index directory and restarting the server.
Error Messages
[.PRPCWorkManager : 0] [STANDARD] [ ] [PegaRULES:07.10] (internal.es.AbstractIndexer) DEBUG - Indexing request failed. Please contact your system administrator.
com.pega.elasticsearch.action.UnavailableShardsException: [work][0] [2] shardIt, [0] active : Timeout waiting for [1m], request: index {[work][def][
at com.pega.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction.raiseTimeoutFailure(TransportShardReplicationOperationAction.java:548)
at com.pega.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$AsyncShardOperationAction$3.onTimeout(TransportShardReplicationOperationAction.java:538)
at com.pega.elasticsearch.cluster.service.InternalClusterService$NotifyTimeout.run(InternalClusterService.java:491)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1156)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:626)
Steps to Reproduce
- Configure Index nodes.
- Specify the Index directory.
- Click Re-Index.
Root Cause
The number of file descriptors defined at the Linux (AIX) level was low. Elastic Search requires numerous file descriptors at the operating system level based on the volume of data. According to the Elastic Search documentation, 32k to 64k is recommended as ulimit.
Resolution
Make the following change to the operating environment:
Set the maximum file limit to 32k or 64k.
Published October 7, 2018 - Updated October 8, 2020
Have a question? Get answers now.
Visit the Collaboration Center to ask questions, engage in discussions, share ideas, and help others.