parfor failed to deserialize data.

Hi,

I tried to run a program using parfor. The code looks like

matlabpool open local 3

parfor i=1:n

..........

end

matlabpool close

where n is about 150000. I have tested the code, when n is small, it works perfectly. However, when I increased n to 150000, it did finished about 2/3 of the parfor loop, the error occur. The error message is

Error using distcompdeserialize Failed to deserialize data.

The machine I used is SunX4100, 4 AMD Opteron and 8Gb memory (2Gb each). The operating system is CentOS 6 and matlab version is R2013a.

Cheers, YP

Answer 1

[Friedrich]

Hi,

it seems like a worker is sending back data to MATLAB which is corruped or too big. keep in mind the size limitation of transfering data between MATLAB and the worker:

64-bit: 2.0 GB

32-bit: 600 MB

Can you share code which raises this error? Do you get that error also when running the code on a different machine?

[Yi-Ping]

Hi Friedrich,

Thanks for your answer. The code is working fine now. I reduce the transfering data and run on other machine with larger memory.

Cheers, YP

[Kalpesh Patil]

HI Yi-Ping, I was also facing same issue about 'deserialize' error. When I use parfor with matlabpool with 2 workers. It works fine, but with 4 it gives error. SO i want to ask how to reduce the transferring data, can you share a bit of code you used for data reduction.