Parfor detailed error message?

7 views (last 30 days)
Simon
Simon on 21 Dec 2015
Commented: Philipp Dreessen on 17 Mar 2017
Hello,
I'm using parfor to run several simulations. After a unknown number of loops the simulation stops:
Error using simulation (line 227)
An UndefinedFunction error was thrown on the workers for 'fetch'. This might
be because the file containing 'fetch' is not accessible on the workers. Use
addAttachedFiles(pool, files) to specify the required files to be attached.
See the documentation for 'parallel.Pool/addAttachedFiles' for more details.
Line 227 is my parfor line, so there is no fetch.
I think I forgot somewhere to close the connection and running to the max. number. But I can't find it. How can I find the fetch function which generates this error? I have about 50 - 100 in my code and don't find the right one.
How can I get a more accurate error message?

Answers (1)

Rebecca Krosnick
Rebecca Krosnick on 23 Dec 2015
How do you create the database connection in your code? Note that when using the Database Toolbox with "parfor", you need to create the database connection on each worker, not just on the client. You should include the database connection statement inside of the "parfor" loop. Since a particular worker may execute multiple iterations of the "parfor", however, you should be sure to create the connection only once per worker. You can do this by creating a "persistent" variable. For example:
parpool
parfor i=1:N
output{i} = myFun(i)
end
where "myFun" is defined as:
function y = myFun(i)
persistent conn
if isempty(conn)
conn = database(...)
end
curs = exec(conn, 'SELECT * FROM myTable'); % This on the workers
curs = fetch(curs);
y = curs.Data;
...
end
On each individual worker, the value of the persistent variable "conn" is retained in memory between calls to "myFun".
Also, are you using a JDBC or ODBC driver? When working with JDBC drivers, you need to make sure that the JDBC driver is actually loaded on all workers, so you may also need to use something like:
pctRunOnAll('javaaddpath(''yourdriver.jar'')')
after having opened the "parpool".
  1 Comment
Philipp Dreessen
Philipp Dreessen on 17 Mar 2017
Thanks a lot, Rebecca! This saved me a lot of time. Especially the fact that I had to load the jdbc driver for each worker was something I didn't know. I had to make a little correction to your code, but below seems to be doing the job.
pctRunOnAll javaaddpath('yourdriver.jar')

Sign in to comment.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!