Compatibility Issue with Kepler K40 GPU and Matlab 2023b

Hello,

I am facing issues with my Nvidia Kepler 40 GPU which I purchased for its fantastic price to performance ratio on double precision computation. The GPU, has a CUDA compute capability of 3.5 and shows up perfectly functional in nvidia-smi and device manager wirthout issue.

However, when I try to call it in gpuDevice in matlab much to my surprise I am greeted with:

>> gpuDevice()

Error using gpuDevice

Graphics driver is out of date. Download and install the latest graphics driver for your GPU from NVIDIA.

Logically, I downloaded and updated to the latest nvidia driver for this gpu as was recommended however I the issue persists. As a sanity check, I tried calling the gpu in an older version of Matlab (2016b) and it works without issue.

What gives? According to the GPU computing requirements for 2023b the card should be supported in this version?

Requirements:

• MATLAB® supports NVIDIA® GPU architectures with compute capability 3.5 to 9.x.
• Install the latest graphics driver. Download drivers for your GPU at NVIDIA Driver Downloads. To see your graphics driver version, use the gpuDevice function.

Please kindly advise me on how to proceed to rectify this issue,

Ivan

[Joss Knight]

What driver did you download? Can you show the output of nvidia-smi?

Answer 1

[Ivan Rodionov]

Hello Joss, I solved the issue, turns out windows was getting creative and installing drivers from 2015! The solution to the problem was to download the newest driver with cuda capability of 11.2 and everything is working fine. An interresting observation turned out that the second gpu I have for an output (the gt620) does not support this newer driver and throws conflicts. The driver that supports both gpus is limited to something like cuda 9 and version 391.35. The version that ended up working was 463.15 which I downloaded via advanced search on windows.

For future reference, a more clear overview of what cuda versions need what would be much appreciated for matlab as the unclear tables in 3 places caused a good deal of confusion.

Thank you and with kind regards,

Ivan

[Joss Knight]

Right Ivan. I was going to say...I was able to get this working with a K40 on both Windows and Linux.

I don't know what language we could use that would be better than "install the latest driver from the NVIDIA website". Everything else we've tried has just led to additional confusion. Yes, your other card is so old that there is no CUDA 11 driver that will recognise it - that is just what happens with old hardware, eventually you can't get a modern driver that supports it.

The only solution with old hardware is to use old software, so that would mean using your older driver and an older version of MATLAB. Sorry about that, but people want MATLAB to support the newest cards and inevitably that means it must drop support for the oldest cards.

[Ivan Rodionov]

Hello Joss,

thank you for getting back to me. I was reffering not to the "install the latest driver from the NVIDIA website" but as a whole to the table. Having 5 different links plus one overview table for only the old version is extremely confusing in my opinion. I would recommend instead to put all the cuda compute capability/architecture/cuda version for matlab version in one table and simply put the newest versions there instead of having 5 different links to facilitate readability and potentially save time.

As for the gpus being "old", this is a separate matter. Normally, I would agree with you, in a free market there is no good use for a 10 year old gpu as there are multiple competetive gpu firms that would produce and constantly innovate architecture aiming for the best price per performance... hahahahaha. No, because of monopolistic practices from NVIDIA, these tesla kepler and some pascal gpus are not only competetive but offer the best! price to performance ratio in terms of double precision floats per euro (from NVIDIA). Not everyone is able to afford a gpu that is worth 10s! of thousands of euros that offers maybe an order of magnitude better performance in double precision BUT costs 2+ orders of magnitude more!!! (This does not even touch on the elephant in the room being AMD and Intel GPUs which matlab conviniently does not support). Lets not kid ourselves here, the only reason a gpu becomes "obsolete" if it offers competetive performance is if it is made obsolete artificially, by practices like these, which I find is disingenuine and insulting to the consumers (I mean seriously, look at the Steam Harware surveys). To give credit where credit is due, I love the software package of Matlab and have been an active user for a number of years now but these policies are unaceptable and how a company dies if it ignores the market trends and implements anti consumer policies such as these (see Blockbuster for reference).

With best regards,

Ivan

[Walter Roberson]

Unfortunately, Nvidia itself drops support for older devices. In order for matlab to continue to provide support, it would need to support multiple generations of tools and automatically select between them. I don't know if that is even feasible (suppose the user has old and new devices in the same system)

[Ivan Rodionov]

Hello Walter,

thank you for your reply. I understand your point and I agree, at the end of the day if you are using CUDA you are limited by NVIDIA and what it can do, but this is also precisely! why everyone else is NOT using CUDA and instead relying on open source tools like Vulkan and sticking to CUDA as an option only. It can be argued that CUDA has marginally better in some circumstances performances but then you can do it like litterally all the other tools on the market and add an option for either CUDA or Vulkan integration as is also industry standard. There is simply no good reason this cannot be done (subsidies from interrested parties do not cound as a good reason not to do this, looking at you NVIDIA). Take for example the clustering tools and SLURM in matlab, clearly if an open source implementation is industry standard and works well it is used, and why wouldn't it be. This was even on the agenda to add support for non NVIDIA gpus but clearly this fell through. Again, dont get me wrong, I enjoy working with matlab and have done so for a number of years, but this situation is unaceptable.

With kind regards,

Ivan

[Walter Roberson]

For better or worse, Mathworks devotes their GPU efforts to following what is happening in Deep Learning research. Use of GPU for other purposes is comparatively incidental for Mathworks.

... and by far most of the Deep Learning reseachers are on Nvidia.

According to a paper I saw about 3 years ago, the second largest share of Deep Learning research is being done on IBM deep learning hardware. Going by memory, it was something like 68% Nvidia, 22% IBM, and everything else together was about 10%. The AMD GPU share for example was about 2%.

Going by those figures... the next target for Mathworks "should" be the IBM hardware. And everything else is practically noise.

The figures have probably changed some with the new Apple M* series systems.

[Joss Knight]

Ivan, would you mind elaborating on what you mean by the 'tables' which confused things for you? Do you mean something in MATLAB's documentation, or on the NVIDIA website?

We've tried hard with our "GPU computing requirements" page to make it difficult to get it wrong, but we're of course happy to get feedback on what more we can do. The problem we had in the past was that advertising the CUDA versions was causing people to download the CUDA toolkit and tie themselves in knots. The idea now is that most people don't need to know anything except that they might need to upgrade their driver. For the odd user with the very latest card or a very old card (Kepler is more than 10 years old!), hopefully all they have to do is double-check the Compute Capability. We've even talked about trying to be more aggressive about saying "any card released since 20xx is supported", which we could do once they've sorted out making sure all their libraries are forward-compatible to new architectures.

As for NVIDIA's aggressive product development plan, it would obviously be nice if NVIDIA could keep supporting older cards and make advances with newer cards without constantly breaking compatibility with older software. But you can't deny that NVIDIA's strategy has worked, since they completely dominate the market (for compute if not graphics). Their strategy has been to aggressively add new hardware features to their cards and to reinvent the ISA each time where necessary. They then write the software to leverage the new features and it ends up being the best software. They could keep support for older cards around, but that would bloat the software massively, and it's already gigabytes, which would cheese people off. So, like I say, their solution is that if you want to use older hardware you can, you just have to use older software.

If it's any consolation I think when it comes to deprecating support for older hardware Apple are probably more aggressive still.

[Ivan Rodionov]

Hello Joss,

thank you for getting back to me. I was reffering to this table: All previous releases: GPU Computing Requirements (R2021b) if I would have designed the website, I would have put this as a single table for all versions or as a table showing the condensed requirements for each version in one place so you dont have to click 5 different things. I do see your point though.

As for NVIDIA marketing plan, it worked swimmingly as they now control the market and have a monopoly. Hopefully in the not too distant future the EU trade commission will antitrust them but we are not here yet. Regarding the software, this is a non issue as this can be effectively implemented using open source APIs where matlab doesnt need to control anything add it as an option as litterally all other industry software does (so Vulkan and CUDA and possibly others) where the user selects which one to use. As for an agressive addition of hardware, this is also highly debatable, as the focus has been on gaming oriented features and sparse calculations while making users needing fp64 compute pay premium by nerfing the architectures on all consumer cards past kepler, which is actually exactly where AMD cards beat them but because of being tied into CUDA matlab does not support these cards.

Finally, apple is a separate matter entirely and it was a real shame watching them go from innovative industry leaders to a mockery of themselves. That said, the new silicon was a step in the right direction.

With kind regards,

Ivan

[Ivan Rodionov]

Hello Walter,

thank you for getting back to me. You are specifically prooving my point about being out of touch with the current market situation by referencing a paper (without a source) from 3 years ago. Getting out of touch with times is exactly! how companies die. Look at the most recent steam hardware survey for an actually representetive dataset of what computers are going to be used. It is about 78% Nvidia, 15% AMD and 7.2% Intel and only! 0.12 percent other devices like IBM and others.

[Joss Knight]

Thanks Ivan. We've retired that table. Now you can only find that at the bottom of the mexcuda documentation. Hopefully the simplicity of the new GPU Computing Requirements doc meets your approval.

[Ivan Rodionov]

Hello Joss,

thank you very much, for putting in the effort to implement an improved table for legiobility. I appreciate it and hope it can save users time and confusion. That said, is it possible the table is inconsistent? I am on 2023b and I have cuda toolkit 11.2 on my driver while the table says 11.8 is needed?

With kind regards and thank you,

Ivan

[Walter Roberson]

@Ivan Rodionov

I don't think we can go by statistics on gaming computers as a proxy for what researchers are using for Deep Learning.

[Ivan Rodionov]

@Walter Roberson

This dataset is extremely representetive as it represents what people are actually using. Sure, there are people with supercomputers using matlab too and on clusters (I've done this myself with the parpool toolbox). BUT, these are not the majority users a percentage of 0.12 is what the market share of these IBM and other devices is in general and I think even if we give it a genererous 5% of users who are in the subset of matlab it is still the minority. The fact of the matter is the median user of the matlab is going to use systems that are similar if slightly better to these, just look how agressively MATLAB is marketed to students!

A good software package is one that is device agnostic, meaning it works equally well as an algorithm on a toaster and on a supercomputer. If a software is arbitrarilly restricted to a subset of devices while literally all other industry software works device agnostically due to using open source libraries it is not a good software.

In fact, this is exacxtly! why the vast majority of users for serious deep learning happen to be using python as it is device agnostic, and is all the acts of how a company misses out on a market by having a warped perception of the target demographic. If these users are already using python for deep learning, the question becomes why use matlab at all and this is where a company dies! Serious deep learning is a highly competetive field and if one company will not do it 10 others will.

I cannot stress enough that these are the attitudes that drive companies into the ground.

[Joss Knight]

MathWorks pays its developers. This puts fundamental limits on what it can do. The fact that MathWorks has to make commercial decisions annoys people, I get it. If it helps, MathWorks has been around since 1984. Whatever decisions it's making don't seem to be 'driving it into the ground'.

[Ivan Rodionov]

Hello @Joss Knight,

Where Matlab is great it is indeed an industry leader and I use it daily myself. Examples being for system design and analysis, competitors (at least the ones I've seen are not at this level yet).

I understand developers have to be paid and the best way to achieve a large revenue is by innovation and becoming an industry leader as this is a recipe for long term sucess. Relying on searching for metaphorical pennies in the sofa by implementing proprietary policies and! providing a subpar tool is not the proper way to do this. This is reflected in the market share of matlab products for deep learning applications and to add insult to injury, Matlab is actively integrating Python libraries into itself to appeal to this market segment... bravo. If making a 180 turn from producing industry leading toolkits to actively positioning yourself as a second rate company in the market via elegant integration of existing toolkits is not the epitome of being run into the ground in this segment, I dont know what is.

A company is a dynamical entity and can end and spring up overnight so age is not a good metric. Just ask Nokia, Kodak or Sears how stagnation went for them...

[Joss Knight]

I too have also run many wildly successful companies in my imagination. It's reality I don't think I'd be very good at it.