Why do I receive an error "java.lang​.OutOfMemo​ryError: unable to create new native thread" while running my parallel jobs on a cluster?

6 views (last 30 days)
I receive the following errors when I try to run my jobs on my cluster:
ERROR: Error using parallel.Job/submit (line 304)
Java exception occurred:
java.lang.OutOfMemoryError: unable to create new native thread
Why do I receive this error and how can I resolve this?

Accepted Answer

MathWorks Support Team
MathWorks Support Team on 3 Nov 2023
Edited: MathWorks Support Team on 3 Nov 2023
On many Linux distributions, by default the process ulimit for a shell is set to a value like 1024. User limits provide control over the resources available to the shell and to processes started by it.
To check this try executing "ulimit -a" command on the terminal and see the "max user processes", i.e. "nproc " field. It is generally set to a number like 1024 by default, so we maY need to increase the value of this field because under the hood Linux count the number of threads which impacts large multi-threaded programs like MATLAB when run in parallel.
To change the value of "max user processes", you can use the following command in Linux shell:
ulimit -u 63536

 

When you check the "ulimit -a" again you should now see the new max user processes. This change may only apply to the active terminal, to make the changes appear session-wide permanently, you can adjust the nproc value in /etc/security/limits.conf, for example, you can add the following line if you wanted to remove the limit from a user named "user":
user hard nproc unlimited

 

More info about how to adjust the ulimit process limit can be found from third-party resources online.
For more information on recommended system limits for Parallel Computing please see: https://www.mathworks.com/help/parallel-computing/recommended-system-limits-for-macintosh-and-linux.html

More Answers (0)

Categories

Find more on Cluster Configuration in Help Center and File Exchange

Tags

No tags entered yet.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!