Support Article
Stream data set generating corrupt files
SA-41258
Summary
Stream dataset generates corrupted files resuliting in data flow failure when reading the files.
Error Messages
com.pega.abc.xyz.impl.commitlog.CommitLogException: Cannot get next entry at com.pega.abc.xyz.impl.commitlog.CommitLog$CommitLogIterator.getNext(CommitLog.java:489) at com.pega.abc.xyz.impl.commitlog.CommitLog$CommitLogIterator.next(CommitLog.java:435) at com.pega.abc.xyz.impl.dataset.stream.StreamBrowseAllRecordsOperation$1.emit(StreamBrowseAllRecordsOperation.java:116) at com.pega.abc.xyz.impl.stream.DataObservableImpl$SafeDataSubscriber.subscribe(DataObservableImpl.java:326) at com.pega.dsm.dnode.impl.stream.DataObservableImpl.subscribe(DataObservableImpl.java:53) at com.pega.dsm.dnode.api.dataflow.DataFlow$2.run(DataFlow.java:374) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1157) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:627) at com.pega.abc.xyz.util.PrpcRunnable$1.run(PrpcRunnable.java:44) at com.pega.abc.xyz.util.PrpcRunnable$1.run(PrpcRunnable.java:41) at com.pega.abc.xyz.util.PrpcRunnable.execute(PrpcRunnable.java:52) at com.pega.abc.xyz.impl.prpc.PrpcThreadFactory$PrpcThread.run(PrpcThreadFactory.java:84) Caused by: com.google.common.util.concurrent.UncheckedExecutionException: com.pega.abc.xyz.impl.commitlog.CommitLogException: Cannot read commit log segment at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2263) at com.google.common.cache.LocalCache.get(LocalCache.java:4000) at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:4004) at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4874) at com.pega.abc.xyz.impl.commitlog.CommitLog.getLastUsedSegment(CommitLog.java:395) at com.pega.abc.xyz.impl.commitlog.CommitLog.access$1200(CommitLog.java:33) at com.pega.abc.xyz.impl.commitlog.CommitLog$CommitLogIterator$1.check(CommitLog.java:478) at com.pega.abc.xyz.impl.commitlog.CommitLog$CommitLogIterator$1.check(CommitLog.java:451) at com.pega.abc.xyz.impl.commitlog.SpinLock.await(SpinLock.java:83) at com.pega.abc.xyz.impl.commitlog.CommitLog$CommitLogIterator.getNext(CommitLog.java:451) ... 11 more Caused by: com.pega.abc.xyz.impl.commitlog.CommitLogException: Cannot read commit log segment at com.pega.abc.xyz.impl.commitlog.CommitLogSegment.openForRead(CommitLogSegment.java:135) at com.pega.abc.xyz.impl.commitlog.CommitLog$1.load(CommitLog.java:157) at com.pega.abc.xyz.impl.commitlog.CommitLog$1.load(CommitLog.java:154) at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3599) at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2379) at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2342) at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2257) ... 20 more Caused by: com.pega.abc.xyz.impl.commitlog.SegmentCorruptedException: Commit log segment file /apps/appserver01/PR_Temp/stream_commit_logs/CC_QUEUE/1392 is corrupted. Position: 32 at com.pega.abc.xyz.impl.commitlog.CommitLogSegment.rewind(CommitLogSegment.java:233) at com.pega.abc.xyz.impl.commitlog.CommitLogSegment.(CommitLogSegment.java:119) at com.pega.abc.xyz.impl.commitlog.CommitLogSegment.openForRead(CommitLogSegment.java:133) ... 26 more
Steps to Reproduce
Unknown
Root Cause
A defect or configuration issue in the operating environment.
The AIX operating system displayed a negative CRC32 result. Therefore, the checksum validation of the files generated by Stream dataset failed sporadically.
Resolution
Apply HFix-35693.
Published January 5, 2019 - Updated October 8, 2020
Have a question? Get answers now.
Visit the Collaboration Center to ask questions, engage in discussions, share ideas, and help others.