2.5 GE, more speed, more headaches

Lord Evermore

Ars Tribunus Militum
2,567
Subscriptor++
Can't believe manufacturers didn't jump on the chance to force such automatic identification into the spec (by hook or by crook) so that you can only buy cables that have an identifier chip built into them like Apple USB cables and people couldn't terminate their own cables anymore and had to hire someone certified to do it which authorizes them to purchase the connectors with the chip in them.
 
  • Like
Reactions: gregorerlich

Demani

Ars Praefectus
5,411
Subscriptor++
I currently can't add VLAN interfaces in MacOS. Who are the idiots in charge of this OS and why are they getting a paycheck??
Which OS version is it? Setting up VLANs has been doable for years and on up to current versions. Is it just the add-on adapter that is missing the option (but you could do it on a built-in interface)?
 

Demani

Ars Praefectus
5,411
Subscriptor++
I'm looking at mine and seeing that now it appears to only allow me to assign a VLAN to the currently connected interface which is not what I thought would have happened (as in, I would have sworn that in the past I could pick any interface I had configured, whether it was connected to the machine or not). Terminal do anything?
ifconfig vlan0 create ifconfig vlan0 vlan 3 vlandev en0 ifconfig vlan0 inet 192.168.126.5 netmask 255.255.255.0
Just curious, and I'm sure you've been down the road (and know more than I do anyway).
 

iljitsch

Ars Tribunus Angusticlavius
9,201
Subscriptor++
Ok we had so much fun here last year that I'm reviving this thread.

The occasion is that I got a new M4 MacBook Air last month, so I want to report on what I ran into on the networking front. It's not all positive but I'm not foaming at the mouth like before. :D

First let's look at the Wi-Fi. My 2013 and 2016 MacBook Pros supported 802.11n / Wi-Fi 4 with 3x3 MIMO for a maximum link speed of 1300 Mbps. That was certainly quite respectable a decade ago.

But it seems like these new MBAs only do 2x2 MIMO, be it at Wi-Fi 6 1200 Mbps. But: that's with 80 MHz channels on the 5 GHz band. Turns out it also supports 160 MHz channels and then the link speed is 2401 Mbps. Wow. I did some testing, and in the same room as the access point, downloads easily top out at 900+ Mbps, the same speed that you get over wired Gigabit Ethernet. (Uploads need some time to get there.)

My access point / home router only has 1 GE ports, but I tried to work around that limitation by doing iperf to another local Mac and Speedtest towards the internet. This topped out at 190 megabytes per second or about 1.5 Gbps. The new MBA also supports the 6 GHz band (Wi-Fi 6E).

So Wi-Fi is very good. What about wired networking? Well, some of my old complaints are still there, with one new one.

As I wrote before, I have two Cable Matters USB-A + attached USB-A-to-USB-C dongle 10/100/1000/2500 adapters. These work very nicely on my old MacBook Pros, although the built-in driver was (is?) terrible, with not even showing you you're at 2.5 GE when you are. And of course Apple has been making installing drivers harder for about two decades now, so that's basically no longer an option.

But with this new Mac, there seems to be some kind of hardware incompatibility: plugging the adapters in using the supplied or a different USB-A-to-USB-C adapter results in... nothing. The blue power light comes on, but not the green and yellow link and activity lights, and the computer doesn't see anything on the USB bus. It's not a driver issue as attaching these though a USB 2.0 hub makes them visible, if not working very reliably. And of course at only a fraction of the supported speeds.

Now my old Apple NICs still work fine: but 10/100 over USB 2 is not what I'm looking for, and the Thunderbolt GE adapter with the TB1 - TB2 adapter is a bit too much.

So I ordered something new. Amazon says the brand is Loydia, but that doesn't show on the actual device or box. This is obviously a no-name Chinese product. The box says it does half duplex and 9k jumboframes. But:

Code:
        supported media:
                media none
                media autoselect
                media 10baseT/UTP mediaopt full-duplex
                media 100baseTX mediaopt full-duplex
                media 1000baseT mediaopt full-duplex
                media 2500Base-T mediaopt full-duplex
                media 5000Base-T mediaopt full-duplex

And System Preferences lets me set the MTU to the enormous size of 1504 bytes. Also no TSO options listed. So this seems to again be a case of Apple including only the barest of bare bones driver for a Realtek chip.

The box has the tinyest QR code ever, which goes to a Realtek driver download page, where you can download MacOS 10.8 - 10.15 drivers. Yeah not even I am running OSes that old.

Still, you get to max out 1 and 2.5 GE speeds without excessive CPU usage, so for the price (14 euros) and the easy plug and play it's still a win.

Strangely the adapter reports 5 GE capability, but trying to get that to work towards my Mac Mini that can go to 10 GE, all the cables I tried ended up at 2.5 GE, and trying to force the Mac Mini to use 5 GE didn't work, as soon as I selected manual and then that speed, it would revert back to automatic configuration at 2.5 GE.
 
  • Like
Reactions: GaitherBill

iljitsch

Ars Tribunus Angusticlavius
9,201
Subscriptor++
As another data point, I have a MBA M2 that a USB ethernet adapter would light up like you described and not register as connected on the first port next to Magsafe, but the other port works perfectly fine.
That's not what's happening here: nothing on either port. It does make sense that the first one might have slightly different behavior, though, as I believe that's the DFU port.

I'm thinking there may be some incompatibility between these dongles and USB/Thunderbolt 4, but whatever.

What would be good though is if we could get away from Ethernet drivers, the same way Apple got rid of printer drivers with AirPrint. Just define a large number of hardware registers that can support all the existing Ethernet functionality, with room for future extensions.

Then, OSes, or perhaps user space code, can interact with those known hardware registers and it's up to the device makers to map that standard functionality to the cool things their hardware can do, with no need to update software drivers ever unless you need to make use of features that exist today but didn't at any point in the past three decades of Ethernet history.
 

BigLan

Ars Tribunus Angusticlavius
7,787
Anyone got a clue why a Realtek 8125 card is struggling to hit full duplex speeds on Win 11?

I added a 10g card to my server and was doing some benchmarks from my PCs with iperf and they can hit 2.5g on either send or receive, but when doing a bidirectional or running two instances the Send will hit 2.5gig but Receive drops to 1-1.5. I'm using the latest drivers from the Realtek site (which must have the slowest download server I've used since dialup days), and turned off power savings / green /energy efficient settings. I'm using -p 8 to create multiple streams and the PCs are 8 core Zen 3 chips and don't look like they're getting stressed.

Network switch is a Unifi Pro Max 16 with the server using SFP+ port, and if I use multiple PCs I can see it doing 7.5G receive but only 4G send so I don't think it's an issue on that end.

Any other settings or tweaks I should check on either the driver or the switch?

(Yeah the 10g is overkill but the empty sfp port was too tempting and Connectx-3 cards are cheap on eBay.)
 

Paladin

Ars Legatus Legionis
33,530
Subscriptor
Anyone got a clue why a Realtek 8125 card is struggling to hit full duplex speeds on Win 11?

I added a 10g card to my server and was doing some benchmarks from my PCs with iperf and they can hit 2.5g on either send or receive, but when doing a bidirectional or running two instances the Send will hit 2.5gig but Receive drops to 1-1.5. I'm using the latest drivers from the Realtek site (which must have the slowest download server I've used since dialup days), and turned off power savings / green /energy efficient settings. I'm using -p 8 to create multiple streams and the PCs are 8 core Zen 3 chips and don't look like they're getting stressed.

Network switch is a Unifi Pro Max 16 with the server using SFP+ port, and if I use multiple PCs I can see it doing 7.5G receive but only 4G send so I don't think it's an issue on that end.

Any other settings or tweaks I should check on either the driver or the switch?

(Yeah the 10g is overkill but the empty sfp port was too tempting and Connectx-3 cards are cheap on eBay.)
Are you using a modern version of iperf3?

I have seen some issues where an old version didn't really fill the link properly.

https://github.com/ar51an/iperf3-win-builds/releases/tag/3.20
 

BigLan

Ars Tribunus Angusticlavius
7,787

w00key

Ars Tribunus Angusticlavius
8,700
Subscriptor
Maybe just PPS limits. With 1 gbps it's easy to hit the max but with higher speeds sometimes it's too much work for the CPU, when the network card keeps yelling at you "hey, packet arrived".

Fancy cards have features like hardware receive and transmit queues so you can dump in a bunch of packets at once, and interrupt coalescing so it doesn't keep interrupting. Or pin interrupts to 1 core (good for cache hits) or spread it to multiple (if 1 core isn't fast enough).

But there are also broken implementations where disabling all these fancy features actually make the card faster. You need to play around and check which setting makes a difference.


Oh and if you control the network, enable jumbo frames. That reduces the packet processing overhead by 75%, key for 10+ Gbps networks.
 
  • Like
Reactions: BigLan

w00key

Ars Tribunus Angusticlavius
8,700
Subscriptor
A simple lack of CPU grunt seems unlikely if we're talking about 8-core Zen 3 systems, though? If I had to guess at a likely culprit it would be driver issues on the client machine - hence the recommendation to test under Linux to get an idea of it's a s/w or h/w problem.
Interrupt handling isn't very efficient. I'm pretty sure that if you switch to Linux and its saner defaults and open drivers the problem would disappear.

But that won't fix the problem on Windows :)


As the issue only appears at 4+ Gbps it smells like a packet overhead issue.
 

BigLan

Ars Tribunus Angusticlavius
7,787
I have jumbo frames enabled already, but suspect it's something in the windows driver. I'll try a Linux live distro when I'm not fixng cars / sick family / work etc but haven't had chance yet.

It's not super urgent to fix - these cards are in my kids PCs and they're unlikely to be moving that much traffic around, but it's just annoyed my inner geek.

And they're Zen 2 chips (3700/3800), but don't think that would make a difference. Checking task manager doesn't show any single core getting hammered but maybe there's driver overhead somewhere, or it's just shoddy Realtek software.
 

steelghost

Ars Tribunus Angusticlavius
6,137
Subscriptor++
As the issue only appears at 4+ Gbps it smells like a packet overhead issue.
Well, it sounds like a "some stuff is being handled on the NIC that should be handled in software because it would be more performant" issue, seeing as the CPU isn't being stressed.

The usual 'fix' is to just disable any kind of offload for anything in the drivers and see what you get. If you get the performance back but the CPU load seems excessive, you can try re-enabling the various offloads one at a time to see what happens.

My son's PC has a Realtek 2.5G interface in it, I'll try and get time to do some iPerf testing on that and see what it looks like (W11 / Zen4, but if you're not CPU limited the different platform shouldn't matter).
 
  • Like
Reactions: BigLan

w00key

Ars Tribunus Angusticlavius
8,700
Subscriptor
  • Like
Reactions: BigLan

BigLan

Ars Tribunus Angusticlavius
7,787
Huh, I did not think B450 would be old enough to have PCIe Gen2 slots. Or maybe the manufacturer had the option of doing 2.0 rather than 3.0 for that generation.
Yeah, I didn't think of it either when I got the cards. Looking at the chipset diagram it's an x4 pcie 3.0 link to the south bridge and then pcie 2.0 lanes from there. The CPU has 20 pcie3 lanes which go 16 to the gpu and 4 for either nvme or a slot. Good old AMD low balling connectivity options.

I should check if the motherboards have 10gb USB ports and maybe switch to that
 

evan_s

Ars Tribunus Angusticlavius
7,314
Subscriptor
Huh, I did not think B450 would be old enough to have PCIe Gen2 slots. Or maybe the manufacturer had the option of doing 2.0 rather than 3.0 for that generation.

The b450 is basically just a rebrand of the b350 for the Zen 2 launch. They threw some software features and better memory/OC in there but basically still the same thing. AMD is horrible about rebranding their chipsets. The 600 vs 800 series chipsets are similar.

B550 adds PCI-E 3x from the chipset and 4x from the CPU lanes while the X570 actually supports PCI-E 4 off the chipset.