Which type of function call provides better performance in MATLAB?

I have 7 different types of function call:

1. An inlined function, where the code author replaces the function call with a copy of the body of the function.

2. A function is defined in a separate MATLAB file. The arguments are passed by the calling function (file-pass).

3. A function is defined in a separate MATLAB file. The arguments are provided by referencing global variables; only indices are provided by the calling function (file-global).

4. A nested function. The arguments are passed by the enclosing function (nest-pass).

5. A nested function. The arguments are those shared with the enclosing function; only indices are provided by the enclosing function (nest-share).

6. A sub function. The arguments are passed by the calling function (sub-pass).

7. A sub function. The arguments are provided by referencing global variables; only indices are provided by the calling function (sub-global).

(For more information, please see the following three MATLAB files: testTop.m, testCompute, and testComputeGlobal.m)

I would like to know which function call provides better performance than the others in general.

Answer 1

[MathWorks Support Team]

The ordering of performance of each function call from the fastest to the slowest tends to be as follows:

inlined > file-pass = nest-pass = sub-pass > nest-share > sub-global > file-global

(A>B means A is faster than B and A=B means A is as fast as B)

First, using an inlined function is the fastest as it does not incur overhead associated with function call.

Second, when the arguments are passed to the callee function, the calling function sets up the arguments in such a way that the callee function knows where to retrieve them. This setup associated with function call in general incurs performance overhead, and therefore file-pass, nest-pass, and sub-pass are slower than inline.

Third, if the workspace is shared with nested functions and the arguments to a nested function are those shared within the workspace, rather than pass-by-value, then performance of that function call is inhibited. If MATLAB sees a shared variable within the shared workspace, it searches the workspace for the variable. On the other hand, if the arguments are passed by the calling function, then MATLAB does not have to search for them. The time taken for this search explains that type nest-share is slower than file-pass, nest-pass, and sub-pass.

Finally, when a function call involves global variables, performance is even more inhibited. This is because to look for global variables, MATLAB has to expand its search space to the outside of the current workspace. Furthermore, the reason a function call involving global variables appears a lot slower than the others is that MATLAB Accelerator does not optimize such a function call. When MATLAB Accelerator is turned off with the following command,

feature accel off

the difference in performance between inline and file-global becomes less significant.

Please note that the behaviors depend largely on various factors such as operating systems, CPU architectures, MATLAB Interpreter, and what the MATLAB code is doing.

[Walter Roberson]

Also note that anonymous function calls are slower than direct function calls, but the difference can vary from almost negligible to being rather substantial in nearly identical code.

(I have some test cases that I put together a couple of years ago, but my system is loaded at the moment so I cannot do timing tests right now.)

Answer 2

[Robert]

Helpful stuff, but shouldn't the first alternative read "1. An Inline function. The body of the function is directly written down (inline)."?

Answer 3

[broken_arrow]

That leaves me a bit confused. Isn't an inlined function the same as just pasting the function code into a script (which would mean a script should be the most performant)? This post on the other hand suggests that functions are generally faster than scripts: https://de.mathworks.com/matlabcentral/answers/415728-details-on-why-functions-are-faster-than-scripts But if I pass variables to a function, the program would still have to look up the variables in the base workspace and launch the function, which creates overhead compared to a script. My "working hypothesis" used to be that everything that is executed only once is put in a script and functions are for code that is used often...

[Stephen23]

"Isn't an inlined function the same as just pasting the function code into a script"

No, the given answer does not specify where that function is called/inlined. It could be in a script, or in another function, or in a class. The original answer states nothing specifically about scripts: your answer (and now my comment) is the only place where the word "script" occurs in this thread.

It is the same as pasting the body of the function somewhere, because that is the definition of inlined code.

"(which would mean a script should be the most performant)"

No. The given answer states that a script with inlined code will be faster than the same script calling the equivalent function. The given answer does not give us any information on the relative speeds of scripts vs. functions.

[broken_arrow]

Ok. So given a set of input data and a set of computations to be performed on these data, which finally returns some output, is there a general best practice guideline on how to arrange the code? As I said, my intuition would be to make a main script since it eliminates function call overheads, and use functions for sections of code that are needed frequently. Of course I could turn the main script into a function, but that shouldn't matter, right?

[Stephen23]

"...given a set of input data and a set of computations to be performed on these data, which finally returns some output, is there a general best practice guideline on how to arrange the code"

Yes: use a function or a class. Avoid scripts.

Personally I see scripts as only useful for experimenting. As soon as I have code that I want to be robust, repeatable, dependable, generalizable, expandible, testable, then it would be converted to a function (with clearly documented interface to the outside world and corresponding test function, in which I collect all relevant test cases).

"...I could turn the main script into a function, but that shouldn't matter"

One provides a clearly documented, testable interface to the world, and isolates their functionality from the calling workspace (which is A Very Good Thing™).

The other doesn't (ugh!)

You state that keeping the "main script" as a script "eliminates function overheads", but I very much doubt that any "function overheads" (whatever they might be) are going to be a significant runtime consumer of your "main script" which only gets call once (or occasionally). Functions are compiled, stored** in a cache for re-use, and have a few other optimizations applied, which makes calling them very fast after the first call.

** Which is why beginners who fill their code with cargo-cult prgramming like CLEAR ALL et al. are doing themselves a large disservice.

[Walter Roberson]

Of course I could turn the main script into a function, but that shouldn't matter, right?

There are some optimizations that are not taken for scripts.

[broken_arrow]

So I should wrap the code into a main function that internally loads the input data, calls other functions to process them and finally returns a struct with all the output data? I can do that of course, and it may indeed have some advantages regarding testing etc.

Let's look at speed alone for the moment: Given that with current versions all code is cached after its first use (which would eliminate the "compilation advantage" of functions), a "function vs. script" speed comparison comes down to whether the script's disadvantage of having to search a bigger workspace outweighs the "overhead" for launching the function (which must exist in some form since inlined functions are the fastest as described above). Is that right?

By the way, if the code is cached after compilation on it's first run, does that mean that compiling into a MEX file will only speed up the first run?

[Rik]

So I should wrap the code into a main function [...]?

Yes.

Let's look at speed alone for the moment [] Is that right?

While it is true that there is an overhead to calling functions, this cost is nearly always worth paying, for the reasons aluded to by Stephen. The compilation for a function will only happen after editing the file (or clear all), while my understanding for scripts is that this will happen every time.

By the way, if the code is cached after compilation on it's first run, does that mean that compiling into a MEX file will only speed up the first run?

The compilation of m-code is unlikely to be to C-code (or equivalent), so mentioning MEX in this context is only confusing matters. Using MEX will generally allow you much faster processing in general, at the cost of doing all the work of implementing your function in C/C++/FORTRAN on your own. On the other hand, m-code will probably be compiled to almost the machine-code level.

[broken_arrow]

The compilation for a function will only happen after editing the file...

So compiled functions are cached throughout the session (unless changed), whereas scripts are recompiled on every run (but loops do not have to be recompiled in every iteration within a run). But in any case, compiled cached functions are recompiled on the first run after Matlab is launched again, right?

The compilation of m-code is unlikely to be to C-code (or equivalent)...

So MEX is basically uncompiled C source code (why is it not compiled like m-code)? And can that really be faster than m-code compiled to almost machine level? Regarding the implementation, turning a function into a MEX file is actually quite easy with the coder.

[Bruno Luong]

"...uncompiled C source code..."

what is that.?

[Bruno Luong]

"The compilation of m-code is unlikely to be to C-code (or equivalent)"

Lately the MATLAB for-loop is so efficient that I'm asking my self whereas that JIT accelerator does not actually compile the code using coder or similar, which is kind of C-code equivalent.

[broken_arrow]

Well, by C source code I mean a .c file as opposed to compiled C code like a .dll . If a MEX file is compiled, why would Matlab compile it differently than a .m file? (I mean, there must be one compilation technique that is most efficient and therefore would always be the best choice)

[Bruno Luong]

"So MEX is basically uncompiled C source code".

Mex is compiled and actually it's actually an DLL on Windows platform, only the file extension is specific.

Don't use the world "uncompiled", it's add confusion. Juts say "C source code".

[Bruno Luong]

(If a MEX file is compiled, why would Matlab compile it differently than a .m file?

Mex is compiled from C-source language

MATLAB is MATLAB language.

They are completely different computer languages (MATLAB is not even a computer language strictly speaking)

I actually don't know what exactly mean an "m-file is compiled". My guess is that is somesort of transformation of the source text file on some proprietary format, possibly faster to convert to CPU instructions than the interpreter run on acscii m-file, p-file or encrypted MCC files (that interpreter step probably occur with scripts), no those details are not published by TMW, and surely change from version to version.

[broken_arrow]

Ok, I think I get. Still seems weird that the coder can go .m -> .c -> MEX, which results in super fast compiled code, but regular compilation of .m files is done in some different way. Or maybe .m files are compiled using some sort of "hybrid method" as you suppose where the coder is used on some parts and others that are not (yet) supported by the coder are compiled in the "usual Matlab" manner.

[Bruno Luong]

".m -> .c -> MEX, which results in super fast compiled code, but regular compilation of .m files is done in some different way."

Again not so fast, you like to extrapolate your conclusion, Nobody actually knows the exact detail (unless if they happens to be TMW employees).

Please read my reply to Rik where I suspect there is perhaps a full compilation to machine language at least for some parts of matlab function with the so called "JIT".

There might be also some politics/commercial reasons (beside technical) why the coder compilation method is not included in standard MATLAB runing engine.

[broken_arrow]

Alright, I think I understand things much better now. Thanks to all of you.

[Rik]

(speculation:) I expect .m-->.c-->MEX will take too long for normal execution to benefit from the investment. It would also require that all parts of your code are actually convertible to C, which is not true, as any Coder report will tell you. Years ago I tried to convert a GUIDE GUI to C to be able to compile it myself to an executable. When I saw I needed to replace half my code I gave up on that.

[Bruno Luong]

" would also require that all parts of your code are actually convertible to C"

Not necessary, it can analyze which part can be compiled which is NOT.

They have the technology, it's call MATLAB CODER, it can analyze the code and then might do some sort of hybrid: half compilation half "interpreters", who knows.

I do some benchmarks on few latest MATLAB version for for-loop vs manual MEX and the for-loop is comparable in speed to the MEX. I can't see how they can achieve such performance without fully compile part of the MATLAB code (when it can).

[Walter Roberson]

A few years ago, the MATLAB "JIT" (Just In Time semi-compiler) was replaced with the "Execution Engine". It has never been clear what the difference is between JIT and Execution Engine, but we can talk a bit about how optimization has changed and that might give us some hints, maybe:

• JIT was much more limited for scripts. Individual execution paths could be optimized, but the overall relationship of the parts could not be studied. Scripts were not cached with JIT
• EE is able to examine the parts of scripts and optimize parts in relationship to other parts
• for example even in scripts now, EE checks to see if there is an obvious assignment to variables, and if not then will treat a name as if it were a function even if it turns out later that something poofs the variable into existance. We had an example yesterday of someone using sim() of a model that did a To Workspace, happening to assign to a name that was also a signal processing function; the signal processing function was called instead because the flow was examined and assumptions were made
• EE is generally a lot less forgiving of poofing variables

For functions, the flow is documented: as soon as the file containing the function is referenced (function called, handle to function is taken), the function is parsed and converted into an internal tokenized form, effectively the same as .p code (except with debugging permitted). A threaded datastructure is created.

Both EE and JIT are known to build internal machine code structures to handle execution of paths. JIT, by its nature, only converts particular branches to machine code as they are traversed. I have not seen any material about whether EE converts everything immediately or only as it traverses it.

There are different JIT strategies. As converting a stretch to machine code has a cost, some JIT developers only convert a block to machine code the second time the path it is executed, under the theory that if a path is only going to be executed once then you might not recover the cost of conversion, but that if you are now executing the path again, that is sufficient evidence to predict that you are likely to be wanting to execute the path again in the future. I do not know if JIT did this "second-time" strategy.

I do not know when the EE does conversion to machine code, whether it is second-time, or first-time, or parse-time. Considering the flow analysis that EE clearly does, I suspect that at the very least, EE works with entire blocks of code -- e.g., at the beginning of a for loop, converting the entire loop instead of waiting for each path in the loop to be encountered. But I do not have any information on how it handles branch prediction and compilation of branches.

Because MATLAB is dynamically typed, any conversion of user functions to machine code cannot be finalized, as the type returned by an expression might change. In theory, MATLAB could pre-analyze the built-ins and the routines provided by Mathworks, to build a dictionary of types each one returns... but I don't think it is doing that, at least not for the .m and .p files (but perhaps for the built-ins). I have some thoughts on how potentially optimization could be handled in the face of changing types, but I have no idea if MATLAB's implementation is anywhere close to my ideas.

Anyhow, what else?

Well, some optimizations are disabled inside try/catch. MATLAB can convert some combinations of statements into calls to BLAS or LAPACK or MKL, but in a try/catch situation you can't do that as much, because at the time of the CATCH you need each variable to have its value as-if flow had proceded statement by statement instead of statements being combined internally. This isn't talked about much -- but I believe the implication is that if you have a try/catch around a bunch of statements, that you can potentially get higher efficiency by moving the statements into a function. Functions are (mostly) black-boxes that either succeed (returning entire results) or fail (in which case none of the output variables are to be assigned to, and you do not have to worry about the segment-to-segment flow. That said, I do not at the moment know what happens with regards to functions that do "in-place" updates of variables but also error() -- if you are working in-place does that mean the output value is however far it got in modification, or is the output defined to be the same as the input if an error occurs? Perhaps in-place is disabled in try/catch... (The debugger will not permit you to examine a variable that it suspects is being modified in-place if you are positioned at the call.)

I have convinced myself that the documentation that says that statements coded "in line" are always fastest... I think that information is wrong, that there are circumstances under which functions can be faster.

[Bruno Luong]

Walter are you speaking as spokeperson for TMW (you use "we") or just as some sort of TMW insider person ? AFAIK you are not TMW employee. You might know some of the inside information, but do you have access to all information?

[Walter Roberson]

"we can talk" --> whoever wants to be involved in the discussion (and can do so without violating nondisclosure). "We" is the correct plural pronoun in English for the case that the subject of the phrase includes the speaker, and is used in "we can talk" when the person making the statement invites common discussion.

"We had an example yesterday" --> public Question https://www.mathworks.com/matlabcentral/answers/779077-error-not-enough-input-arguments-for-half-model-vehicle-pitch-angle . As it was posted to the public, it would have been misleading for me to say that "I" was presented with the question. The question was presented to us collectively; I am simply the person who happened to respond, using my knowledge of a change that was posted in the Release Notes a couple of releases ago.

Those were the only two places that I used "we" in my earlier comment.

If I had access to the source code, I would would have said "I have not checked" or "I haven't had time to research" instead of all those places where I said I do not know or that I have not seen any information. Or I would have used my hypothetical source access to read up on the implementation and then described it... or refrained from describing it if disclosure would have violated Non-disclosure.

When I write that JIT did not do something but EE does, I describe what we could observe at the time; I am not describing what the JIT execution could have done if it had been further developed. From my position on the outside, it is not clear that EE is anything more than a rebranding and tinkering of JIT, but twice staff have corrected me to say that JIT is not used anymore but Execution Engine is now used. What is the difference? We (the public) do not know, but we (the public) can examine the documentation of the changes since JIT and hope to reconstruct differences in technology.

[Bruno Luong]

"A few years ago, the MATLAB "JIT" (Just In Time semi-compiler) was replaced with the "Execution Engine". It has never been clear what the difference is between JIT and Execution Engine,"

According to Loren's blog, JIT is not replaced by EE, not it's actually integrated into EE since R2015b

"MATLAB has employed JIT compilation since release R13 in 2002. The original JIT compiler was designed as a supplement to the MATLAB interpreter, and supported a limited subset of the language. The new execution engine was designed with a focus on JIT compilation, where JIT compilation is an integral part of the engine and able to compile the entire MATLAB language."

So JIT still exists in theory.

[Walter Roberson]

Interesting, at https://www.mathworks.com/content/dam/mathworks/mathworks-dot-com/images/events/matlabexpo/kr/2016/matlab-programming-techniques-for-efficiency-performance.pdf I find the description

MATLAB Execution Engine

Old system had two different execution mechanisms –a JIT and an Interpreter. New system has a single execution mechanism.

Old JIT was designed for FORTRAN-like constructs within MATLAB. New JIT is designed for the entire MATLAB language.

Old system had a monolithic architecture that was difficult to extend. New system has a Modular, Thread-safe, and Platform re-targetable architecture.

[Bruno Luong]

My own reading of such description is that

• old JIT is separated to EE and consisiting of calling prebuilt library to replace MATLAB code,
• new JIT is integrated into EE and is close to a compilation to native machine instructions.