java.io.NotSerializableException while writing to Oracle Coherence cache - preloading

Question

I have 2 storage enabled cache nodes that i am trying to use for pre-loading of cache. About 1 million accounts are to be loaded by these 2 storage enabled nodes. Both key and value are String objects which I am trying to write to cache. I am using InvocationService.execute() method to invoke the pre-loading tasks asynchronously:
for (Map.Entry<Member, LoaderInvocable> entry : mappedWork.entrySet()) {
Member member = entry.getKey();
LoaderInvocable task = entry.getValue();
invocationService.execute(task, Collections.singleton(member), null);
}
LoaderInvocable is a class that is implementing Invocable and Serializable interfaces and its run() method has been overridden to performs the actual work of reading from database and writing to the cache.
InvocationService is defined as below in the coherence config file:
<invocation-scheme>
<scheme-name>
InvocationScheme</scheme-name>
<service-name>
LoaderInvocationService</service-name>
<autostart system-property="tangosol.coherence.invocation.autostart">true</autostart>
</invocation-scheme>
Below is the exception that i am getting:
2016-02-22 17:16:24,612 [pool-1-thread-1] ERROR (support.context.SessionExecutable) Caught exception from SessionExecutable.execute()
(Wrapped) java.io.NotSerializableException: com.oracle.common.collections.ConcurrentLinkedQueue
at com.tangosol.util.Base.ensureRuntimeException(Base.java:289)
at com.tangosol.util.Base.ensureRuntimeException(Base.java:270)
at com.tangosol.coherence.component.util.daemon.queueProcessor.packetProcessor.PacketPublisher.packetizeMessage(PacketPublisher.CDB:28)
at com.tangosol.coherence.component.util.daemon.queueProcessor.packetProcessor.PacketPublisher$InQueue.add(PacketPublisher.CDB:8)
at com.tangosol.coherence.component.util.daemon.queueProcessor.packetProcessor.PacketPublisher.post(PacketPublisher.CDB:1)
at com.tangosol.coherence.component.net.Message.dispatch(Message.CDB:77)
at com.tangosol.coherence.component.net.Message.post(Message.CDB:1)
at com.tangosol.coherence.component.util.daemon.queueProcessor.service.Grid.post(Grid.CDB:2)
at com.tangosol.coherence.component.util.daemon.queueProcessor.service.Grid.send(Grid.CDB:1)
at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.InvocationService.execute(InvocationService.CDB:33)
at com.tangosol.coherence.component.util.safeService.SafeInvocationService.execute(SafeInvocationService.CDB:1)
.
.
.
.
.
.
Caused by: java.io.NotSerializableException: com.oracle.common.collections.ConcurrentLinkedQueue
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1180)
at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1528)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1493)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1416)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1174)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:346)
at java.util.Hashtable.writeObject(Hashtable.java:988)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:975)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1480)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1416)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1174)
at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1528)
at java.io.ObjectOutputStream.defaultWriteObject(ObjectOutputStream.java:438)
at com.tangosol.coherence.Component.writeObject(Component.CDB:1)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:975)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1480)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1416)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1174)
at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1528)
at java.io.ObjectOutputStream.defaultWriteObject(ObjectOutputStream.java:438)
at com.tangosol.coherence.Component.writeObject(Component.CDB:1)
It seems that half of the accounts have been cached successfully. Can it be node specific issue? Both of the storage enabled cache nodes are on the same server using the same cluster configuration. From the logs it is clear that both nodes successfully joined the cluster.

Answer 1

Thanks Praveen. I have implemented the PortableObject interface and not facing the NotSerializableException anymore. But now i am facing a new problem. The second node is not invoking the task and leaving the cluster without any exception in the logs.
I used the InvocationObserver which suggests that memberLeft() the cluster. Can it be something wrong with my implementation of readExternal() and writeExternal() methods for serialization? Below is the implementation:
#Override
public void readExternal(PofReader paramPofReader) throws IOException {
// TODO Auto-generated method stub
cacheName = paramPofReader.readString(0);
firstRow = paramPofReader.readLong(1);
lastRow = paramPofReader.readLong(2);
}
#Override
public void writeExternal(PofWriter paramPofWriter) throws IOException {
// TODO Auto-generated method stub
paramPofWriter.writeString(0, cacheName);
paramPofWriter.writeLong(1, firstRow);
paramPofWriter.writeLong(2, lastRow);
}