Support Article
Deserialization issues with DNodes - DSM
SA-28641
Summary
When a DDS system is configured with a mix of DNode and non-DNode, systems attempts to save a record to a DDS Dataset from a non-DNode system are redirected to a DNode system using Hazelcast distributed executors. When DNode trieds to deserialie the data from the non DNode, the following deserialization error will occur.
Error Messages
2016-09-22 00:00:07,741 [c-operation.thread-0] [ STANDARD] [ ] [ ] (ns.MemberCallableTaskOperation) FATAL - [<an ip address>]:5701 [c45e0b33d0fdcc58a9dae0a6c417afac] [3.4.1] java.io.IOException: Could not properly deserialize the record
com.hazelcast.nio.serialization.HazelcastSerializationException: java.io.IOException: Could not properly deserialize the record
at com.hazelcast.nio.serialization.SerializationServiceImpl.handleException(SerializationServiceImpl.java:419)
at com.hazelcast.nio.serialization.SerializationServiceImpl.toObject(SerializationServiceImpl.java:266)
at com.hazelcast.spi.impl.NodeEngineImpl.toObject(NodeEngineImpl.java:186)
at com.hazelcast.executor.impl.operations.BaseCallableTaskOperation.getCallable(BaseCallableTaskOperation.java:71)
at com.hazelcast.executor.impl.operations.BaseCallableTaskOperation.beforeRun(BaseCallableTaskOperation.java:53)
at com.hazelcast.spi.impl.BasicOperationService$OperationHandler.handle(BasicOperationService.java:727)
at com.hazelcast.spi.impl.BasicOperationService$OperationHandler.access$500(BasicOperationService.java:709)
at com.hazelcast.spi.impl.BasicOperationService$OperationPacketHandler.handle(BasicOperationService.java:683)
at com.hazelcast.spi.impl.BasicOperationService$OperationPacketHandler.handle(BasicOperationService.java:627)
at com.hazelcast.spi.impl.BasicOperationService$OperationPacketHandler.access$1500(BasicOperationService.java:614)
at com.hazelcast.spi.impl.BasicOperationService$BasicDispatcherImpl.dispatch(BasicOperationService.java:566)
at com.hazelcast.spi.impl.BasicOperationScheduler$OperationThread.process(BasicOperationScheduler.java:466)
at com.hazelcast.spi.impl.BasicOperationScheduler$OperationThread.doRun(BasicOperationScheduler.java:458)
at com.hazelcast.spi.impl.BasicOperationScheduler$OperationThread.run(BasicOperationScheduler.java:432)
Caused by:
java.io.IOException: Could not properly deserialize the record
at com.pega.dsm.dnode.api.data.Record.readObject(Record.java:226)
at sun.reflect.GeneratedMethodAccessor178.invoke(Unknown Source)
Steps to Reproduce
Startup process described above
Root Cause
When a DDS system is configured with a mix of DNode and non-DNode systems attempts to save a record to a DDS Dataset from a non-DNode system are redirected to a DNode system using Hazelcast distributed executors. Remote execution requires serialization and deserialization of the records to be saved.
The deserialization process reads the properties of the records based on a preceding length. The deserialization method uses ObjectInputStream read(byte[]) which is not guaranteed to read all of the data in one attempt and hence for records of any size the data read is smaller than the data expected. The read(byte[]) should be replaced with readFully(byte[]) to ensure the byte array will be read in its entirety
Resolution
Apply HFix-29714.
Published October 7, 2016 - Updated October 8, 2020
Have a question? Get answers now.
Visit the Collaboration Center to ask questions, engage in discussions, share ideas, and help others.