Is a 10 gigabit ethernet worth it for a matlab cluster?

3 views (last 30 days)
Hi there,
I am planning some updates on an existing cluster with quad core desktops running Matlab. The current desktops have gigabit network adapters and are connected to a gigabit switch. Speeds are limited to 100 megabytes/s. Would I see any benefits from an upgrade to 10 gigabit network on distributed processing?
Acquisition of high performance desktops is out of question, since we only have low-mid end computers (i7 4790). Therefore we are working on a future proof solution that enhances distributed processing on existing PC's.
Thanks!

Accepted Answer

Jan
Jan on 21 Nov 2017
Edited: Jan on 21 Nov 2017
Would I see any benefits from an upgrade to 10 gigabit network on
distributed processing?
If your distributed computations are limited by the speed of the connections:
Yes. Current bandwidth: 1 GBit, planned bandwidth: 10 GBit, expected speedup is up to factor 10.
If the inter-process communication is not the bottleneck of your codes:
No. If you use less than 1 GBit, accelerating the connecting has no effect.
  3 Comments
Jan
Jan on 21 Nov 2017
Hi Paulo: If you are running low on budget, it is obligatory to get a reliable estimation of what you need. Therefore it is obligatory to know which problems you are working on. 1.8 sec delay might be huge or tiny, depending on what the real workloads are. Maybe there is an interprocess communication, such that running on a local machine is much faster, or the complete communication consists of providing the inputs and outputs only. Maybe the delay of 1.8 seconds is coming from accessing shared resources, or from the network speed.
I do not know an efficient and easy way to monitor this. I would ask a professional consultant, who checks the requirements personally. The public forum and a rough description of the machines and the timing of a (test?) case is not sufficient.
Paulo Ribeiro
Paulo Ribeiro on 21 Nov 2017
Jan, thanks! Low budget requires carefull thinking. Nice picture btw. Best regards, Paulo.

Sign in to comment.

More Answers (1)

Walter Roberson
Walter Roberson on 21 Nov 2017
You should probably use SNMP to pull some usage statistics off of your existing gigabit switch.
If your existing gigabit switch does not support SNMP or similar methods of measuring load (a "managed switch") then it is not suitable for professional use.
What make and model is the existing switch?
Back when gigabit switches were new and we were trying to figure out whether we needed to upgrade from 100 megabit, I looked into switches a fair bit and tested what I could. What I found at the time was that quality of implementation mattered a lot, that switches that were nominally rated faster were sometimes slower for most practical use. It was the time when gigabit chips were just coming onto the consumer market, and there was pressure to buy one of the inexpensive gigabit switches because of the lower price, but my testbench showed that some of those were slower than the 100 megabit switches we already had. I would worry about the same kind of issues for 10 gigabit.
How many ports do you need? What kind of connector would you need? Do you need an uplink port, and if so what kind? Do you have 10 gigabit interfaces on the existing computers?
I see some fairly high prices once you get into the higher port counts. For lower port count, at first glance this one looks plausible: https://www.servethehome.com/netgear-prosafe-xs708t-review-quiet-8-port-10gbase-t-switch/ but I cannot make out from the diagrams whether traffic information is available off of it.
  5 Comments
Paulo Ribeiro
Paulo Ribeiro on 21 Nov 2017
Thanks, Walter. Would you suggest any appropriate benchmark? Perhaps a code sample?
Walter Roberson
Walter Roberson on 21 Nov 2017
Some of your own code is always the best benchmark.

Sign in to comment.

Categories

Find more on Cluster Configuration in Help Center and File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!