From: kernel.org-fo5k2w@ycharbi.fr
To: "Jacob Keller" <jacob.e.keller@intel.com>,
kernel.org-fo5k2w@ycharbi.fr, "Jeff Daly" <jeffd@silicom-usa.com>
Cc: anthony.l.nguyen@intel.com, intel-wired-lan@lists.osuosl.org,
jesse.brandeburg@intel.com, netdev@vger.kernel.org,
regressions@leemhuis.info
Subject: Re: Non-functional ixgbe driver between Intel X553 chipset and Cisco switch via kernel >=6.1 under Debian
Date: Thu, 09 May 2024 13:26:37 +0000 [thread overview]
Message-ID: <9eaa16ac88aafb6dee36c5781ae4de7881bb03a2@ycharbi.fr> (raw)
In-Reply-To: <4a0bf7cf-d108-49ac-ac7c-6136a070c44b@intel.com>
Hi,
> No link detected, but it does detect this is a 10GBaseT cable.
> Interesting it doesn't report FEC or autonegotiation. Hmm.
In fact, I personally find it strange that the "Supported link modes" is "10000baseT/Full". A DAC is not a SFP+ 8P8C (RJ45) module. Wouldn't it be more logical if the modes reported were the same as those obtained by an "ethtool eth2" on the Connectx-3 side? :
Settings for eth2:
Supported ports: [ FIBRE ]
Supported link modes: 10000baseKX4/Full
1000baseX/Full
10000baseCR/Full
10000baseSR/Full
Supported pause frame use: Symmetric Receive-only
Supports auto-negotiation: No
Supported FEC modes: Not reported
Advertised link modes: 10000baseKX4/Full
1000baseX/Full
10000baseCR/Full
10000baseSR/Full
Advertised pause frame use: Symmetric
Advertised auto-negotiation: No
Advertised FEC modes: Not reported
Speed: 10000Mb/s
Duplex: Full
Auto-negotiation: off
Port: Direct Attach Copper
PHYAD: 0
Transceiver: internal
Supports Wake-on: d
Wake-on: d
Current message level: 0x00000014 (20)
link ifdown
Link detected: yes
In other words, isn't the fact that the reported mode is "10000baseT/Full" a bug in itself?
> Knowing the kernel is the important part, we don't have specific
> versioning of drivers in the kernel anymore.
Ok. I take note of this information.
> The steps would require that you build the kernel manually. I can
> outline the steps i would take here
>
> 1. get the kernel source from git.kernel.org. I place it in $HOME/git/linux
> 2. switch to v5.10 with 'git switch --detach v5.10'
> 2. copy the debian 5.10 config file to $HOME/git/linux/.config
> 3. build kernel with 'make -j24' (adjust -j depending on how much CPU
> you want to spend building the kernel)
> 4. install with 'sudo make -j24 modules_install && sudo make install'
> 5. reboot and select the v5.10 kernel, double check it works.
> 6. in $HOME/git/linux run 'git bisect start' to initiate the bisect session.
> 7. First, label the current v5.10 commit as good with 'git bisect good'
> 8. Second, label the v6.1 commit as bad with 'git bisect bad v6.1'
>
> This will initiate a bisect session and will checkout the kernel
> approximately halfway between v5.10 and v6.1. For each bisection point
> it checks, run the following steps:
>
> 1. 'make olddefconfig' to update the configuration for this version
> 2. 'make -j24' to rebuild with the current version
> 3. 'sudo make -j24 modules_install && sudo make install' to install this
> version.
> 4. reboot into that version and check its behavior.
> 5. If it works properly then run 'git bisect good'
> 6. If it works incorrectly, then run 'git bisect bad'
>
> A new commit will be selected. It will pick one between the latest good
> point and the closest bad point, essentially honing in towards the
> incorrect behavior.
>
> If for any reason a commit can't be built or tested, you can use "git
> bisect skip" and it will skip around a bit to find another point that
> can be tried.
Thank you for your and Thorsten Leemhuis's advice. I don't know whether the following Bisect log will be of any help to you. However, I have determined precisely that the problem was introduced with version 6.1. If I boot into 6.0, it works perfectly. So there are fewer differences to search for the problem. Here's the feedback from Bisect, but I'm still dubious about the relevance of this log because the “git bisect bad v6.1” command returned "7614896350aa20764c5eca527262d9eb0a57da63 était à la fois good et bad"... I didn't really understand how it all worked... :
git bisect start
# good: [4fe89d07dcc2804c8b562f6c7896a45643d34b2f] Linux 6.0
git bisect good 45eb8ae5370d5df1ee8236f45df3f29103ba6e12
# bad: [830b3c68c1fb1e9176028d02ef86f3cf76aa2476] Linux 6.1
git bisect bad 7614896350aa20764c5eca527262d9eb0a57da63
I should point out that I had to switch back to Debian 11 because 12 and 13 refuse to compile these old kernels... Anyway, I compiled the versions successively and came across the difference in operation between 6.0 and 6.1.
> I suspect those changes must have broken the Cisco switch link behavior.
> I unfortunately do not know enough about this hardware or the SFI
> configuration to understand why this causes it.
>
> If you don't want to try bisect, I would suggest trying to revert that
> commit or simply replace the ixgbe_setup_sfi_x550a function with the one
> from out-of-tree here. If you do that, you can rebuild just ixgbe with
> "make M=drivers/net/ethernet/intel/ixgbe" and then insert the module
> with "insmod drivers/net/ethernet/intel/ixgbe/ixgbe.ko".
>
> It seems likely that this change had unintended side effect which broke
> the Cisco switch linking.
If I do a "git revert 565736048bd5f9888990569993c6b6bfdf6dcb6d" to go back before the state of the suspected problem commit, compile kernel 6.1 and boot on it, it works perfectly.
So it turns out that this is the source of the malfunction and was introduced with Linux 6.1.
> I've added Jeff Daly, in the hopes that he could provide more details on
> the change.
>
> @Jeff, it seems likely that the change you made at 565736048bd5 ("ixgbe:
> Manual AN-37 for troublesome link partners for X550 SFI") is breaking
> some other switches. It would help if you could shed some light on this
> change as otherwise we might need to revert it and once again break the
> setup you fixed.
>
> Thanks,
> Jake
Let me know if you need more information. I'll be happy to help!
Best regards.
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ Yohan Charbi
⢿⡄⠘⠷⠚⠋⠀ Cordialement
⠈⠳⣄⠀
next prev parent reply other threads:[~2024-05-09 13:26 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-02-15 11:02 Non-functional ixgbe driver between Intel X553 chipset and Cisco switch via kernel >=6.1 under Debian kernel.org-fo5k2w
2024-05-03 13:01 ` kernel.org-fo5k2w
2024-05-03 18:37 ` Jacob Keller
2024-05-04 13:29 ` kernel.org-fo5k2w
2024-05-06 21:18 ` Jacob Keller
2024-05-07 6:31 ` [Intel-wired-lan] " Linux regression tracking (Thorsten Leemhuis)
2024-05-21 1:59 ` Jeff Daly
2024-05-09 13:26 ` kernel.org-fo5k2w [this message]
2024-05-21 0:02 ` Jacob Keller
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=9eaa16ac88aafb6dee36c5781ae4de7881bb03a2@ycharbi.fr \
--to=kernel.org-fo5k2w@ycharbi.fr \
--cc=anthony.l.nguyen@intel.com \
--cc=intel-wired-lan@lists.osuosl.org \
--cc=jacob.e.keller@intel.com \
--cc=jeffd@silicom-usa.com \
--cc=jesse.brandeburg@intel.com \
--cc=netdev@vger.kernel.org \
--cc=regressions@leemhuis.info \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).