NanoPi R5S | M.2 NVMe SSD randomly not detected

[eggcllnt]

Issue Description

After installing DietPi on my NanoPi R5S, the M.2 SSD that was previously detected and working properly under FriendlyWRT is no longer visible. The device doesn't appear in lsblk output, despite being physically installed and previously functional.

System Information

Device: NanoPi R5S
Original OS: FriendlyWRT (SSD was detected)
Current OS: DietPi
Kernel version: 6.6.56-current-rockchip64

Steps Taken

Verified SSD isn't showing up in basic disk utilities:

dietpi@DietPi:~$ lsblk
NAME         MAJ:MIN RM  SIZE RO TYPE MOUNTPOINTS
mmcblk1      179:0    0 28.9G  0 disk 
|-mmcblk1p1  179:1    0  1.1G  0 part 
`-mmcblk1p2  179:2    0    1M  0 part 
mmcblk1boot0 179:32   0    4M  1 disk 
mmcblk1boot1 179:64   0    4M  1 disk 
mmcblk0      179:96   0 59.6G  0 disk 
`-mmcblk0p1  179:97   0 59.6G  0 part /

Checked if NVMe kernel modules are available:

dietpi@DietPi:~$ find /lib/modules/$(uname -r) -name "*nvme*"
/lib/modules/6.6.56-current-rockchip64/kernel/drivers/nvme
/lib/modules/6.6.56-current-rockchip64/kernel/drivers/nvme/target/nvmet.ko
/lib/modules/6.6.56-current-rockchip64/kernel/drivers/nvme/target/nvmet-tcp.ko
/lib/modules/6.6.56-current-rockchip64/kernel/drivers/nvme/target/nvme-fcloop.ko
/lib/modules/6.6.56-current-rockchip64/kernel/drivers/nvme/target/nvme-loop.ko
/lib/modules/6.6.56-current-rockchip64/kernel/drivers/nvme/target/nvmet-fc.ko
/lib/modules/6.6.56-current-rockchip64/kernel/drivers/nvme/host/nvme-fabrics.ko
/lib/modules/6.6.56-current-rockchip64/kernel/drivers/nvme/host/nvme-tcp.ko
/lib/modules/6.6.56-current-rockchip64/kernel/drivers/nvme/host/nvme-fc.ko
/lib/modules/6.6.56-current-rockchip64/kernel/drivers/nvmem
/lib/modules/6.6.56-current-rockchip64/kernel/drivers/nvmem/nvmem-rmem.ko
/lib/modules/6.6.56-current-rockchip64/kernel/drivers/nvmem/nvmem_qcom-spmi-sdam.ko

Checked if NVMe modules are loaded:

dietpi@DietPi:~$ lsmod | grep -E 'nvme|ahci|sata|scsi'
[No output]

Tried manually loading NVMe modules:

dietpi@DietPi:~$ sudo modprobe nvme
dietpi@DietPi:~$ sudo modprobe nvme-core
dietpi@DietPi:~$ lsblk
[SSD still not detected]

Examined PCIe status in dmesg:

dietpi@DietPi:~$ sudo dmesg | grep -i -E 'pcie|nvme'
[    0.847519] rockchip-dw-pcie 3c0000000.pcie: host bridge /pcie@fe260000 ranges:
[    0.847617] rockchip-dw-pcie 3c0000000.pcie:       IO 0x00f4100000..0x00f41fffff -> 0x00f4100000
[    0.847669] rockchip-dw-pcie 3c0000000.pcie:      MEM 0x00f4200000..0x00f5ffffff -> 0x00f4200000
[    0.847709] rockchip-dw-pcie 3c0000000.pcie:      MEM 0x0300000000..0x033fffffff -> 0x0040000000
[    0.847994] rockchip-dw-pcie 3c0000000.pcie: iATU: unroll T, 8 ob, 8 ib, align 64K, limit 8G
[    1.055022] rockchip-dw-pcie 3c0000000.pcie: PCIe Gen.2 x1 link up
[    1.055776] rockchip-dw-pcie 3c0000000.pcie: PCI host bridge to bus 0000:00
[    1.082499] pcieport 0000:00:00.0: PME: Signaling with IRQ 22
[    1.083143] pcieport 0000:00:00.0: AER: enabled with IRQ 22
[    2.191280] rockchip-dw-pcie 3c0400000.pcie: host bridge /pcie@fe270000 ranges:
[    2.191359] rockchip-dw-pcie 3c0400000.pcie:       IO 0x00f2100000..0x00f21fffff -> 0x00f2100000
[    2.191429] rockchip-dw-pcie 3c0400000.pcie:      MEM 0x00f2200000..0x00f3ffffff -> 0x00f2200000
[    2.191469] rockchip-dw-pcie 3c0400000.pcie:      MEM 0x0340000000..0x037fffffff -> 0x0040000000
[    2.203145] rockchip-dw-pcie 3c0400000.pcie: iATU: unroll T, 8 ob, 8 ib, align 64K, limit 8G
[    2.410930] rockchip-dw-pcie 3c0400000.pcie: PCIe Gen.2 x1 link up
[    2.411180] rockchip-dw-pcie 3c0400000.pcie: PCI host bridge to bus 0001:00
[    2.434544] pcieport 0001:00:00.0: PME: Signaling with IRQ 73
[    2.435110] pcieport 0001:00:00.0: AER: enabled with IRQ 73
[    2.436425] rockchip-dw-pcie 3c0800000.pcie: host bridge /pcie@fe280000 ranges:
[    2.436498] rockchip-dw-pcie 3c0800000.pcie:       IO 0x00f0100000..0x00f01fffff -> 0x00f0100000
[    2.436550] rockchip-dw-pcie 3c0800000.pcie:      MEM 0x00f0200000..0x00f1ffffff -> 0x00f0200000
[    2.436590] rockchip-dw-pcie 3c0800000.pcie:      MEM 0x0380000000..0x03bfffffff -> 0x0040000000
[    2.446555] rockchip-dw-pcie 3c0800000.pcie: iATU: unroll T, 8 ob, 8 ib, align 64K, limit 8G
[    3.551144] rockchip-dw-pcie 3c0800000.pcie: Phy link never came up
[    3.551360] rockchip-dw-pcie 3c0800000.pcie: PCI host bridge to bus 0002:00
[    3.564057] pcieport 0002:00:00.0: PME: Signaling with IRQ 75
[    3.564570] pcieport 0002:00:00.0: AER: enabled with IRQ 75

Checked available PCI devices:

dietpi@DietPi:~$ sudo lspci
0000:00:00.0 PCI bridge: Rockchip Electronics Co., Ltd RK3568 Remote Signal Processor (rev 01)
0000:01:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller (rev 05)
0001:00:00.0 PCI bridge: Rockchip Electronics Co., Ltd RK3568 Remote Signal Processor (rev 01)
0001:01:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller (rev 05)
0002:00:00.0 PCI bridge: Rockchip Electronics Co., Ltd RK3568 Remote Signal Processor (rev 01)

Checked available PCIe and NVMe-related device tree overlays:

dietpi@DietPi:~$ find /boot/ -name "*nvme*.dtbo" -o -name "*m2*.dtbo" -o -name "*pcie*.dtbo"
/boot/dtb-6.6.56-current-rockchip64/rockchip/overlay/rockchip-rk3568-hk-pwm2.dtbo
/boot/dtb-6.6.56-current-rockchip64/rockchip/overlay/rockchip-rk3399-pcie-gen2.dtbo

Modified kernel parameters in DietPi configuration to try enabling PCIe:

extraargs=net.ifnames=0 pcie_aspm=off pci=nomsi

After reboot, the issue persists with the same error message:

[    3.539542] rockchip-dw-pcie 3c0800000.pcie: Phy link never came up

Observations

The M.2 SSD worked properly under FriendlyWRT on the same hardware
NVMe kernel modules are available in DietPi but not automatically loaded
Manually loading NVMe modules does not resolve the issue
PCIe port 3c0800000.pcie reports "Phy link never came up" which appears to be the port connected to the M.2 slot
Two RTL8125 Ethernet controllers are properly detected on PCIe buses 0000:00 and 0001:00
No device appears to be detected on PCIe bus 0002:00, which is likely the one connected to the M.2 slot

Questions

Are there specific device tree configurations needed for NanoPi R5S M.2 slot in DietPi?
Is there a known issue with DietPi's PCIe initialization for the NanoPi R5S M.2 slot?
Are there specific boot parameters or overlays that need to be enabled for M.2 NVMe support?

Any assistance would be greatly appreciated.

[MichaIng]

I pushed new kernel and bootloader packages to our APT repo. Could you try it with those:

sudo apt update
sudo apt upgrade
sudo /boot/dietpi/func/dietpi-set_hardware flash-uboot-mmc
sudo reboot

[eggcllnt]

@MichaIng thanks for looking into that issue!
I've updated my existing system with the instractions that you've provided, but there were no result. Same missing nvme drive.

I've reinstalled system from scratch. It worked as expected, booted, updated, but no nvme again. One weird thing happened is it was rebooted and after ethernet ports were unresponsive. so, I've powered off, then powered on and things booted as usual asking to optout data collection. No nvme though.

edit: forgot to mention, yes, I ran /boot/dietpi/func/dietpi-set_hardware flash-uboot-mmc after reinstall.

I've added to /boot/dietpiEnv.txt extraargs pci=realloc pcie_aspm=off params, rebooted and it showed up. But this behaviour I was observed and previously.

On next reboot it disappear.

logs from successful boot

root@DietPi:~# lsblk
NAME         MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
mmcblk1      179:0    0  28.9G  0 disk 
└─mmcblk1p1  179:1    0  28.9G  0 part /
mmcblk1boot0 179:32   0     4M  1 disk 
mmcblk1boot1 179:64   0     4M  1 disk 
nvme0n1      259:0    0 465.8G  0 disk 
└─nvme0n1p1  259:1    0 465.8G  0 part

root@DietPi:~# dmesg | grep -i -E 'pcie|nvme'
[    0.000000] Kernel command line: root=UUID=43620ab3-751b-4f33-91a2-14ca0a8027d3 rootfstype=ext4 rootwait console=ttyS2,1500000 console=tty1 consoleblank=0 coherent_pool=2M usb-storage.quirks= net.ifnames=0 pci=realloc pcie_aspm=off
[    0.000000] PCIe ASPM is disabled
[    0.146022] /pcie@fe260000: Fixed dependency cycle(s) with /pcie@fe260000/legacy-interrupt-controller
[    0.174869] /pcie@fe270000: Fixed dependency cycle(s) with /pcie@fe270000/legacy-interrupt-controller
[    0.175797] /pcie@fe280000: Fixed dependency cycle(s) with /pcie@fe280000/legacy-interrupt-controller
[    1.979652] rockchip-dw-pcie 3c0000000.pcie: host bridge /pcie@fe260000 ranges:
[    1.979752] rockchip-dw-pcie 3c0000000.pcie:       IO 0x00f4100000..0x00f41fffff -> 0x00f4100000
[    1.979805] rockchip-dw-pcie 3c0000000.pcie:      MEM 0x00f4200000..0x00f5ffffff -> 0x00f4200000
[    1.979847] rockchip-dw-pcie 3c0000000.pcie:      MEM 0x0300000000..0x033fffffff -> 0x0040000000
[    1.980111] rockchip-dw-pcie 3c0000000.pcie: iATU: unroll T, 8 ob, 8 ib, align 64K, limit 8G
[    2.176964] rockchip-dw-pcie 3c0000000.pcie: PCIe Gen.2 x1 link up
[    2.177683] rockchip-dw-pcie 3c0000000.pcie: PCI host bridge to bus 0000:00
[    2.177920] pci 0000:00:00.0: [1d87:3566] type 01 class 0x060400 PCIe Root Port
[    2.187782] pci 0000:01:00.0: [10ec:8125] type 00 class 0x020000 PCIe Endpoint
[    2.193190] pcieport 0000:00:00.0: PME: Signaling with IRQ 22
[    2.193768] pcieport 0000:00:00.0: AER: enabled with IRQ 22
[    4.171029] rockchip-dw-pcie 3c0400000.pcie: host bridge /pcie@fe270000 ranges:
[    4.171793] rockchip-dw-pcie 3c0400000.pcie:       IO 0x00f2100000..0x00f21fffff -> 0x00f2100000
[    4.172629] rockchip-dw-pcie 3c0400000.pcie:      MEM 0x00f2200000..0x00f3ffffff -> 0x00f2200000
[    4.173586] rockchip-dw-pcie 3c0400000.pcie:      MEM 0x0340000000..0x037fffffff -> 0x0040000000
[    4.182905] rockchip-dw-pcie 3c0400000.pcie: iATU: unroll T, 8 ob, 8 ib, align 64K, limit 8G
[    4.382822] rockchip-dw-pcie 3c0400000.pcie: PCIe Gen.2 x1 link up
[    4.383749] rockchip-dw-pcie 3c0400000.pcie: PCI host bridge to bus 0001:00
[    4.388854] pci 0001:00:00.0: [1d87:3566] type 01 class 0x060400 PCIe Root Port
[    4.411384] pci 0001:01:00.0: [10ec:8125] type 00 class 0x020000 PCIe Endpoint
[    4.429076] pcieport 0001:00:00.0: PME: Signaling with IRQ 79
[    4.430234] pcieport 0001:00:00.0: AER: enabled with IRQ 79
[    4.451745] rockchip-dw-pcie 3c0800000.pcie: host bridge /pcie@fe280000 ranges:
[    4.452473] rockchip-dw-pcie 3c0800000.pcie:       IO 0x00f0100000..0x00f01fffff -> 0x00f0100000
[    4.453756] rockchip-dw-pcie 3c0800000.pcie:      MEM 0x00f0200000..0x00f1ffffff -> 0x00f0200000
[    4.454583] rockchip-dw-pcie 3c0800000.pcie:      MEM 0x0380000000..0x03bfffffff -> 0x0040000000
[    4.463797] rockchip-dw-pcie 3c0800000.pcie: iATU: unroll T, 8 ob, 8 ib, align 64K, limit 8G
[    4.853010] rockchip-dw-pcie 3c0800000.pcie: PCIe Gen.1 x1 link up
[    4.853877] rockchip-dw-pcie 3c0800000.pcie: PCI host bridge to bus 0002:00
[    4.857686] pci 0002:00:00.0: [1d87:3566] type 01 class 0x060400 PCIe Root Port
[    4.880205] pci 0002:01:00.0: [1344:5416] type 00 class 0x010802 PCIe Endpoint
[    4.882880] pci 0002:01:00.0: 2.000 Gb/s available PCIe bandwidth, limited by 2.5 GT/s PCIe x1 link at 0002:00:00.0 (capable of 31.504 Gb/s with 8.0 GT/s PCIe x4 link)
[    4.899225] pcieport 0002:00:00.0: enabling device (0000 -> 0002)
[    4.904000] pcieport 0002:00:00.0: PME: Signaling with IRQ 82
[    4.905363] pcieport 0002:00:00.0: AER: enabled with IRQ 82
[    4.907149] nvme nvme0: pci function 0002:01:00.0
[    4.907656] nvme 0002:01:00.0: enabling device (0000 -> 0002)
[    5.035223] nvme nvme0: allocated 64 MiB host memory buffer.
[    5.142951] nvme nvme0: 4/0/0 default/read/poll queues
[    5.163016]  nvme0n1: p1

root@DietPi:~# lspci
0000:00:00.0 PCI bridge: Rockchip Electronics Co., Ltd RK3568 Remote Signal Processor (rev 01)
0000:01:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller (rev 05)
0001:00:00.0 PCI bridge: Rockchip Electronics Co., Ltd RK3568 Remote Signal Processor (rev 01)
0001:01:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller (rev 05)
0002:00:00.0 PCI bridge: Rockchip Electronics Co., Ltd RK3568 Remote Signal Processor (rev 01)
0002:01:00.0 Non-Volatile memory controller: Micron Technology Inc 2550 NVMe SSD (DRAM-less) (rev 01)

root@DietPi:~# lspci -vvv

...

0002:01:00.0 Non-Volatile memory controller: Micron Technology Inc 2550 NVMe SSD (DRAM-less) (rev 01) (prog-if 02 [NVM Express])
        Subsystem: Micron Technology Inc 2550 NVMe SSD (DRAM-less)
        Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0
        Interrupt: pin A routed to IRQ 82
        Region 0: Memory at f0200000 (64-bit, non-prefetchable) [size=16K]
        Capabilities: [80] Express (v2) Endpoint, MSI 00
                DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s unlimited, L1 unlimited
                        ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ SlotPowerLimit 0W
                DevCtl: CorrErr+ NonFatalErr+ FatalErr+ UnsupReq+
                        RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+ FLReset-
                        MaxPayload 128 bytes, MaxReadReq 512 bytes
                DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend-
                LnkCap: Port #1, Speed 8GT/s, Width x4, ASPM L1, Exit Latency L1 unlimited
                        ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
                LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk-
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 2.5GT/s (downgraded), Width x1 (downgraded)
                        TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
                DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ NROPrPrP- LTR+
                         10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt+ EETLPPrefix-
                         EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
                         FRS- TPHComp- ExtTPHComp-
                         AtomicOpsCap: 32bit- 64bit- 128bitCAS-
                DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR+ 10BitTagReq- OBFF Disabled,
                         AtomicOpsCtl: ReqEn-
                LnkCap2: Supported Link Speeds: 2.5-8GT/s, Crosslink- Retimer- 2Retimers- DRS-
                LnkCtl2: Target Link Speed: 8GT/s, EnterCompliance- SpeedDis-
                         Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
                         Compliance Preset/De-emphasis: -6dB de-emphasis, 0dB preshoot
                LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete- EqualizationPhase1-
                         EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
                         Retimer- 2Retimers- CrosslinkRes: unsupported
        Capabilities: [d0] MSI-X: Enable+ Count=33 Masked-
                Vector table: BAR=0 offset=00002000
                PBA: BAR=0 offset=00003000
        Capabilities: [e0] MSI: Enable- Count=1/32 Maskable+ 64bit+
                Address: 0000000000000000  Data: 0000
                Masking: 00000000  Pending: 00000000
        Capabilities: [f8] Power Management version 3
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
                Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [100 v1] Latency Tolerance Reporting
                Max snoop latency: 0ns
                Max no snoop latency: 0ns
        Capabilities: [110 v1] L1 PM Substates
                L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1+ ASPM_L1.2+ ASPM_L1.1+ L1_PM_Substates+
                          PortCommonModeRestoreTime=10us PortTPowerOnTime=50us
                L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1-
                           T_CommonMode=0us LTR1.2_Threshold=0ns
                L1SubCtl2: T_PwrOn=10us
        Capabilities: [128 v1] Alternative Routing-ID Interpretation (ARI)
                ARICap: MFVC- ACS-, Next Function: 0
                ARICtl: MFVC- ACS-, Function Group: 0
        Capabilities: [1e0 v1] Data Link Feature <?>
        Capabilities: [200 v2] Advanced Error Reporting
                UESta:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                UEMsk:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                UESvrt: DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
                CESta:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr-
                CEMsk:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
                AERCap: First Error Pointer: 00, ECRCGenCap- ECRCGenEn- ECRCChkCap+ ECRCChkEn-
                        MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
                HeaderLog: 00000000 00000000 00000000 00000000
        Capabilities: [300 v1] Secondary PCI Express
                LnkCtl3: LnkEquIntrruptEn- PerformEqu-
                LaneErrStat: 0
        Kernel driver in use: nvme

logs from unsuccessful boot

root@DietPi:~# lsblk
NAME         MAJ:MIN RM  SIZE RO TYPE MOUNTPOINTS
mmcblk1      179:0    0 28.9G  0 disk 
└─mmcblk1p1  179:1    0 28.9G  0 part /
mmcblk1boot0 179:32   0    4M  1 disk 
mmcblk1boot1 179:64   0    4M  1 disk 

dmesg | grep -i pci
[    0.000000] Kernel command line: root=UUID=43620ab3-751b-4f33-91a2-14ca0a8027d3 rootfstype=ext4 rootwait console=ttyS2,1500000 console=tty1 consoleblank=0 coherent_pool=2M usb-storage.quirks= net.ifnames=0 pci=realloc pcie_aspm=off
[    0.000000] PCIe ASPM is disabled
[    0.146022] /pcie@fe260000: Fixed dependency cycle(s) with /pcie@fe260000/legacy-interrupt-controller
[    0.174869] /pcie@fe270000: Fixed dependency cycle(s) with /pcie@fe270000/legacy-interrupt-controller
[    0.175797] /pcie@fe280000: Fixed dependency cycle(s) with /pcie@fe280000/legacy-interrupt-controller
[    0.905287] PCI: CLS 0 bytes, default 64
[    1.979652] rockchip-dw-pcie 3c0000000.pcie: host bridge /pcie@fe260000 ranges:
[    1.979752] rockchip-dw-pcie 3c0000000.pcie:       IO 0x00f4100000..0x00f41fffff -> 0x00f4100000
[    1.979805] rockchip-dw-pcie 3c0000000.pcie:      MEM 0x00f4200000..0x00f5ffffff -> 0x00f4200000
[    1.979847] rockchip-dw-pcie 3c0000000.pcie:      MEM 0x0300000000..0x033fffffff -> 0x0040000000
[    1.980111] rockchip-dw-pcie 3c0000000.pcie: iATU: unroll T, 8 ob, 8 ib, align 64K, limit 8G
[    2.176964] rockchip-dw-pcie 3c0000000.pcie: PCIe Gen.2 x1 link up
[    2.177683] rockchip-dw-pcie 3c0000000.pcie: PCI host bridge to bus 0000:00
[    2.177743] pci_bus 0000:00: root bus resource [bus 00-0f]
[    2.177777] pci_bus 0000:00: root bus resource [io  0x0000-0xfffff] (bus address [0xf4100000-0xf41fffff])
[    2.177811] pci_bus 0000:00: root bus resource [mem 0xf4200000-0xf5ffffff]
[    2.177839] pci_bus 0000:00: root bus resource [mem 0x300000000-0x33fffffff] (bus address [0x40000000-0x7fffffff])
[    2.177920] pci 0000:00:00.0: [1d87:3566] type 01 class 0x060400 PCIe Root Port
[    2.177980] pci 0000:00:00.0: ROM [mem 0x00000000-0x0000ffff pref]
[    2.178009] pci 0000:00:00.0: PCI bridge to [bus 01-ff]
[    2.178039] pci 0000:00:00.0:   bridge window [io  0x0000-0x0fff]
[    2.178068] pci 0000:00:00.0:   bridge window [mem 0x00000000-0x000fffff]
[    2.178101] pci 0000:00:00.0:   bridge window [mem 0x00000000-0x000fffff 64bit pref]
[    2.178203] pci 0000:00:00.0: supports D1 D2
[    2.178231] pci 0000:00:00.0: PME# supported from D0 D1 D3hot
[    2.187589] pci_bus 0000:01: busn_res: can not insert [bus 01-ff] under [bus 00-0f] (conflicts with (null) [bus 00-0f])
[    2.187782] pci 0000:01:00.0: [10ec:8125] type 00 class 0x020000 PCIe Endpoint
[    2.187896] pci 0000:01:00.0: BAR 0: initial BAR value 0x00000000 invalid
[    2.187924] pci 0000:01:00.0: BAR 0 [io  size 0x0100]
[    2.188035] pci 0000:01:00.0: BAR 2 [mem 0x00000000-0x0000ffff 64bit]
[    2.188118] pci 0000:01:00.0: BAR 4 [mem 0x00000000-0x00003fff 64bit]
[    2.188709] pci 0000:01:00.0: supports D1 D2
[    2.188735] pci 0000:01:00.0: PME# supported from D0 D1 D2 D3hot D3cold
[    2.189744] pci_bus 0000:00: max bus depth: 1 pci_try_num: 2
[    2.189803] pci 0000:00:00.0: bridge window [mem 0xf4200000-0xf42fffff]: assigned
[    2.189840] pci 0000:00:00.0: ROM [mem 0xf4300000-0xf430ffff pref]: assigned
[    2.189873] pci 0000:00:00.0: bridge window [io  0x1000-0x1fff]: assigned
[    2.189908] pci 0000:01:00.0: BAR 2 [mem 0xf4200000-0xf420ffff 64bit]: assigned
[    2.189984] pci 0000:01:00.0: BAR 4 [mem 0xf4210000-0xf4213fff 64bit]: assigned
[    2.190060] pci 0000:01:00.0: BAR 0 [io  0x1000-0x10ff]: assigned
[    2.190102] pci 0000:00:00.0: PCI bridge to [bus 01-ff]
[    2.190129] pci 0000:00:00.0:   bridge window [io  0x1000-0x1fff]
[    2.190158] pci 0000:00:00.0:   bridge window [mem 0xf4200000-0xf42fffff]
[    2.190192] pci_bus 0000:00: resource 4 [io  0x0000-0xfffff]
[    2.190217] pci_bus 0000:00: resource 5 [mem 0xf4200000-0xf5ffffff]
[    2.190242] pci_bus 0000:00: resource 6 [mem 0x300000000-0x33fffffff]
[    2.190268] pci_bus 0000:01: resource 0 [io  0x1000-0x1fff]
[    2.190292] pci_bus 0000:01: resource 1 [mem 0xf4200000-0xf42fffff]
[    2.193190] pcieport 0000:00:00.0: PME: Signaling with IRQ 22
[    2.193768] pcieport 0000:00:00.0: AER: enabled with IRQ 22
[    4.171029] rockchip-dw-pcie 3c0400000.pcie: host bridge /pcie@fe270000 ranges:
[    4.171793] rockchip-dw-pcie 3c0400000.pcie:       IO 0x00f2100000..0x00f21fffff -> 0x00f2100000
[    4.172629] rockchip-dw-pcie 3c0400000.pcie:      MEM 0x00f2200000..0x00f3ffffff -> 0x00f2200000
[    4.173586] rockchip-dw-pcie 3c0400000.pcie:      MEM 0x0340000000..0x037fffffff -> 0x0040000000
[    4.182905] rockchip-dw-pcie 3c0400000.pcie: iATU: unroll T, 8 ob, 8 ib, align 64K, limit 8G
[    4.382822] rockchip-dw-pcie 3c0400000.pcie: PCIe Gen.2 x1 link up
[    4.383749] rockchip-dw-pcie 3c0400000.pcie: PCI host bridge to bus 0001:00
[    4.384466] pci_bus 0001:00: root bus resource [bus 00-0f]
[    4.385168] pci_bus 0001:00: root bus resource [io  0x100000-0x1fffff] (bus address [0xf2100000-0xf21fffff])
[    4.387144] pci_bus 0001:00: root bus resource [mem 0xf2200000-0xf3ffffff]
[    4.387835] pci_bus 0001:00: root bus resource [mem 0x340000000-0x37fffffff] (bus address [0x40000000-0x7fffffff])
[    4.388854] pci 0001:00:00.0: [1d87:3566] type 01 class 0x060400 PCIe Root Port
[    4.389662] pci 0001:00:00.0: ROM [mem 0x00000000-0x0000ffff pref]
[    4.390263] pci 0001:00:00.0: PCI bridge to [bus 01-ff]
[    4.390765] pci 0001:00:00.0:   bridge window [io  0x0000-0x0fff]
[    4.391326] pci 0001:00:00.0:   bridge window [mem 0x00000000-0x000fffff]
[    4.391953] pci 0001:00:00.0:   bridge window [mem 0x00000000-0x000fffff 64bit pref]
[    4.392744] pci 0001:00:00.0: supports D1 D2
[    4.393284] pci 0001:00:00.0: PME# supported from D0 D1 D3hot
[    4.410135] pci_bus 0001:01: busn_res: can not insert [bus 01-ff] under [bus 00-0f] (conflicts with (null) [bus 00-0f])
[    4.411384] pci 0001:01:00.0: [10ec:8125] type 00 class 0x020000 PCIe Endpoint
[    4.412144] pci 0001:01:00.0: BAR 0 [io  0x0000-0x00ff]
[    4.412750] pci 0001:01:00.0: BAR 2 [mem 0x00000000-0x0000ffff 64bit]
[    4.413452] pci 0001:01:00.0: BAR 4 [mem 0x00000000-0x00003fff 64bit]
[    4.414773] pci 0001:01:00.0: supports D1 D2
[    4.415198] pci 0001:01:00.0: PME# supported from D0 D1 D2 D3hot D3cold
[    4.416783] pci_bus 0001:00: max bus depth: 1 pci_try_num: 2
[    4.417401] pci 0001:00:00.0: bridge window [mem 0xf2200000-0xf22fffff]: assigned
[    4.418107] pci 0001:00:00.0: ROM [mem 0xf2300000-0xf230ffff pref]: assigned
[    4.418759] pci 0001:00:00.0: bridge window [io  0x100000-0x100fff]: assigned
[    4.419425] pci 0001:01:00.0: BAR 2 [mem 0xf2200000-0xf220ffff 64bit]: assigned
[    4.420154] pci 0001:01:00.0: BAR 4 [mem 0xf2210000-0xf2213fff 64bit]: assigned
[    4.420878] pci 0001:01:00.0: BAR 0 [io  0x100000-0x1000ff]: assigned
[    4.421556] pci 0001:00:00.0: PCI bridge to [bus 01-ff]
[    4.422051] pci 0001:00:00.0:   bridge window [io  0x100000-0x100fff]
[    4.422655] pci 0001:00:00.0:   bridge window [mem 0xf2200000-0xf22fffff]
[    4.422677] pci_bus 0001:00: resource 4 [io  0x100000-0x1fffff]
[    4.422689] pci_bus 0001:00: resource 5 [mem 0xf2200000-0xf3ffffff]
[    4.424402] pci_bus 0001:00: resource 6 [mem 0x340000000-0x37fffffff]
[    4.424415] pci_bus 0001:01: resource 0 [io  0x100000-0x100fff]
[    4.424425] pci_bus 0001:01: resource 1 [mem 0xf2200000-0xf22fffff]
[    4.429076] pcieport 0001:00:00.0: PME: Signaling with IRQ 79
[    4.430234] pcieport 0001:00:00.0: AER: enabled with IRQ 79
[    4.451745] rockchip-dw-pcie 3c0800000.pcie: host bridge /pcie@fe280000 ranges:
[    4.452473] rockchip-dw-pcie 3c0800000.pcie:       IO 0x00f0100000..0x00f01fffff -> 0x00f0100000
[    4.453756] rockchip-dw-pcie 3c0800000.pcie:      MEM 0x00f0200000..0x00f1ffffff -> 0x00f0200000
[    4.454583] rockchip-dw-pcie 3c0800000.pcie:      MEM 0x0380000000..0x03bfffffff -> 0x0040000000
[    4.463797] rockchip-dw-pcie 3c0800000.pcie: iATU: unroll T, 8 ob, 8 ib, align 64K, limit 8G
[    4.853010] rockchip-dw-pcie 3c0800000.pcie: PCIe Gen.1 x1 link up
[    4.853877] rockchip-dw-pcie 3c0800000.pcie: PCI host bridge to bus 0002:00
[    4.854536] pci_bus 0002:00: root bus resource [bus 00-0f]
[    4.855051] pci_bus 0002:00: root bus resource [io  0x200000-0x2fffff] (bus address [0xf0100000-0xf01fffff])
[    4.855943] pci_bus 0002:00: root bus resource [mem 0xf0200000-0xf1ffffff]
[    4.856571] pci_bus 0002:00: root bus resource [mem 0x380000000-0x3bfffffff] (bus address [0x40000000-0x7fffffff])
[    4.857686] pci 0002:00:00.0: [1d87:3566] type 01 class 0x060400 PCIe Root Port
[    4.858385] pci 0002:00:00.0: BAR 0 [mem 0x00000000-0x3fffffff]
[    4.858937] pci 0002:00:00.0: BAR 1 [mem 0x00000000-0x3fffffff]
[    4.859483] pci 0002:00:00.0: ROM [mem 0x00000000-0x0000ffff pref]
[    4.860050] pci 0002:00:00.0: PCI bridge to [bus 00]
[    4.860512] pci 0002:00:00.0:   bridge window [io  0x0000-0x0fff]
[    4.861115] pci 0002:00:00.0:   bridge window [mem 0x00000000-0x000fffff]
[    4.861758] pci 0002:00:00.0:   bridge window [mem 0x00000000-0x000fffff 64bit pref]
[    4.862568] pci 0002:00:00.0: supports D1 D2
[    4.862972] pci 0002:00:00.0: PME# supported from D0 D1 D3hot
[    4.879078] pci 0002:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[    4.880205] pci 0002:01:00.0: [1344:5416] type 00 class 0x010802 PCIe Endpoint
[    4.881091] pci 0002:01:00.0: BAR 0 [mem 0x00000000-0x00003fff 64bit]
[    4.882880] pci 0002:01:00.0: 2.000 Gb/s available PCIe bandwidth, limited by 2.5 GT/s PCIe x1 link at 0002:00:00.0 (capable of 31.504 Gb/s with 8.0 GT/s PCIe x4 link)
[    4.885064] pci_bus 0002:01: busn_res: [bus 01-0f] end is updated to 01
[    4.885747] pci_bus 0002:00: max bus depth: 1 pci_try_num: 2
[    4.886329] pci 0002:00:00.0: BAR 0 [mem 0x380000000-0x3bfffffff]: assigned
[    4.886999] pci 0002:00:00.0: BAR 1 [mem size 0x40000000]: can't assign; no space
[    4.887699] pci 0002:00:00.0: BAR 1 [mem size 0x40000000]: failed to assign
[    4.888423] pci 0002:00:00.0: bridge window [mem 0xf0200000-0xf02fffff]: assigned
[    4.890677] pci 0002:00:00.0: ROM [mem 0xf0300000-0xf030ffff pref]: assigned
[    4.891397] pci 0002:01:00.0: BAR 0 [mem 0xf0200000-0xf0203fff 64bit]: assigned
[    4.892151] pci 0002:00:00.0: PCI bridge to [bus 01]
[    4.892628] pci 0002:00:00.0:   bridge window [mem 0xf0200000-0xf02fffff]
[    4.893349] pci_bus 0002:00: No. 2 try to assign unassigned res
[    4.893955] pci 0002:00:00.0: BAR 1 [mem size 0x40000000]: can't assign; no space
[    4.894656] pci 0002:00:00.0: BAR 1 [mem size 0x40000000]: failed to assign
[    4.895316] pci 0002:00:00.0: PCI bridge to [bus 01]
[    4.895797] pci 0002:00:00.0:   bridge window [mem 0xf0200000-0xf02fffff]
[    4.896445] pci_bus 0002:00: resource 4 [io  0x200000-0x2fffff]
[    4.897049] pci_bus 0002:00: resource 5 [mem 0xf0200000-0xf1ffffff]
[    4.897657] pci_bus 0002:00: resource 6 [mem 0x380000000-0x3bfffffff]
[    4.898259] pci_bus 0002:01: resource 1 [mem 0xf0200000-0xf02fffff]
[    4.899225] pcieport 0002:00:00.0: enabling device (0000 -> 0002)
[    4.904000] pcieport 0002:00:00.0: PME: Signaling with IRQ 82
[    4.905363] pcieport 0002:00:00.0: AER: enabled with IRQ 82
[    4.907149] nvme nvme0: pci function 0002:01:00.0

[StephanStS]

My R5S (2 GB RAM) with my NVMe works: Booting from an 8 GB SD card and using the NVMe as a mounted ext4 drive.

lsblk give my nvme0n1 disk:

root@dietpi:~# lsblk
NAME         MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
mmcblk1      179:0    0   7,3G  0 disk
└─mmcblk1p1  179:1    0   3,7G  0 part /
mmcblk1boot0 179:32   0     4M  1 disk
mmcblk1boot1 179:64   0     4M  1 disk
nvme0n1      259:0    0 238,5G  0 disk
└─nvme0n1p1  259:1    0 238,5G  0 part /mnt/nvme_1

Latest kernel:

root@dietpi:~# uname -r
6.12.25-current-rockchip64

lspci gives the NVMe controller:

root@dietpi:~# lspci
0000:00:00.0 PCI bridge: Rockchip Electronics Co., Ltd RK3568 Remote Signal Processor (rev 01)
0000:01:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller (rev 05)
0001:00:00.0 PCI bridge: Rockchip Electronics Co., Ltd RK3568 Remote Signal Processor (rev 01)
0001:01:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller (rev 05)
0002:00:00.0 PCI bridge: Rockchip Electronics Co., Ltd RK3568 Remote Signal Processor (rev 01)
0002:01:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller SM981/PM981/PM983

dmesg output:

root@dietpi:~# dmesg | grep -i -E 'pcie|nvme'
[    0.143610] /pcie@fe260000: Fixed dependency cycle(s) with /pcie@fe260000/legacy-interrupt-controller
[    0.171861] /pcie@fe270000: Fixed dependency cycle(s) with /pcie@fe270000/legacy-interrupt-controller
[    0.172754] /pcie@fe280000: Fixed dependency cycle(s) with /pcie@fe280000/legacy-interrupt-controller
[    1.957517] rockchip-dw-pcie 3c0000000.pcie: host bridge /pcie@fe260000 ranges:
[    1.957634] rockchip-dw-pcie 3c0000000.pcie:       IO 0x00f4100000..0x00f41fffff -> 0x00f4100000
[    1.957686] rockchip-dw-pcie 3c0000000.pcie:      MEM 0x00f4200000..0x00f5ffffff -> 0x00f4200000
[    1.957725] rockchip-dw-pcie 3c0000000.pcie:      MEM 0x0300000000..0x033fffffff -> 0x0040000000
[    1.958067] rockchip-dw-pcie 3c0000000.pcie: iATU: unroll T, 8 ob, 8 ib, align 64K, limit 8G
[    2.157522] rockchip-dw-pcie 3c0000000.pcie: PCIe Gen.2 x1 link up
[    2.158257] rockchip-dw-pcie 3c0000000.pcie: PCI host bridge to bus 0000:00
[    2.158484] pci 0000:00:00.0: [1d87:3566] type 01 class 0x060400 PCIe Root Port
[    2.168331] pci 0000:01:00.0: [10ec:8125] type 00 class 0x020000 PCIe Endpoint
[    2.181011] pcieport 0000:00:00.0: PME: Signaling with IRQ 22
[    2.181680] pcieport 0000:00:00.0: AER: enabled with IRQ 22
[    4.223495] rockchip-dw-pcie 3c0800000.pcie: host bridge /pcie@fe280000 ranges:
[    4.224220] rockchip-dw-pcie 3c0800000.pcie:       IO 0x00f0100000..0x00f01fffff -> 0x00f0100000
[    4.225044] rockchip-dw-pcie 3c0800000.pcie:      MEM 0x00f0200000..0x00f1ffffff -> 0x00f0200000
[    4.225924] rockchip-dw-pcie 3c0800000.pcie:      MEM 0x0380000000..0x03bfffffff -> 0x0040000000
[    4.235450] rockchip-dw-pcie 3c0800000.pcie: iATU: unroll T, 8 ob, 8 ib, align 64K, limit 8G
[    4.433517] rockchip-dw-pcie 3c0800000.pcie: PCIe Gen.3 x1 link up
[    4.433815] rockchip-dw-pcie 3c0800000.pcie: PCI host bridge to bus 0002:00
[    4.433934] pci 0002:00:00.0: [1d87:3566] type 01 class 0x060400 PCIe Root Port
[    4.444752] pci 0002:01:00.0: [144d:a808] type 00 class 0x010802 PCIe Endpoint
[    4.445729] pci 0002:01:00.0: 7.876 Gb/s available PCIe bandwidth, limited by 8.0 GT/s PCIe x1 link at 0002:00:00.0 (capable of 31.504 Gb/s with 8.0 GT/s PCIe x4 link)
[    4.456727] pcieport 0002:00:00.0: PME: Signaling with IRQ 79
[    4.458124] pcieport 0002:00:00.0: AER: enabled with IRQ 79
[    4.459238] nvme nvme0: pci function 0002:01:00.0
[    4.459283] nvme 0002:01:00.0: enabling device (0000 -> 0002)
[    4.462345] nvme nvme0: missing or invalid SUBNQN field.
[    4.726175] nvme nvme0: D3 entry latency set to 8 seconds
[    4.767112] nvme nvme0: 4/0/0 default/read/poll queues
[    4.809665]  nvme0n1: p1
[    4.816234] rockchip-dw-pcie 3c0400000.pcie: host bridge /pcie@fe270000 ranges:
[    4.820954] rockchip-dw-pcie 3c0400000.pcie:       IO 0x00f2100000..0x00f21fffff -> 0x00f2100000
[    4.831095] rockchip-dw-pcie 3c0400000.pcie:      MEM 0x00f2200000..0x00f3ffffff -> 0x00f2200000
[    4.831133] rockchip-dw-pcie 3c0400000.pcie:      MEM 0x0340000000..0x037fffffff -> 0x0040000000
[    4.833987] rockchip-dw-pcie 3c0400000.pcie: iATU: unroll T, 8 ob, 8 ib, align 64K, limit 8G
[    5.045558] rockchip-dw-pcie 3c0400000.pcie: PCIe Gen.2 x1 link up
[    5.050706] rockchip-dw-pcie 3c0400000.pcie: PCI host bridge to bus 0001:00
[    5.075141] pci 0001:00:00.0: [1d87:3566] type 01 class 0x060400 PCIe Root Port
[    5.130178] pci 0001:01:00.0: [10ec:8125] type 00 class 0x020000 PCIe Endpoint
[    5.232502] pcieport 0001:00:00.0: PME: Signaling with IRQ 86
[    5.237642] pcieport 0001:00:00.0: AER: enabled with IRQ 86
[   17.615598] EXT4-fs (nvme0n1p1): recovery complete
[   17.615735] EXT4-fs (nvme0n1p1): mounted filesystem 0cd73e32-a067-4358-ba5f-9716bf0d2231 r/w with ordered data mode. Quota mode: none.

Tree overlays gives more:

root@dietpi:~# find /boot/ -name "*nvme*.dtbo" -o -name "*m2*.dtbo" -o -name "*pcie*.dtbo"
/boot/dtb-6.12.25-current-rockchip64/rockchip/overlay/rockchip-rk3399-pcie-gen2.dtbo
/boot/dtb-6.12.25-current-rockchip64/rockchip/overlay/rockchip-rk3588-pwm0-m2.dtbo
/boot/dtb-6.12.25-current-rockchip64/rockchip/overlay/rockchip-rk3568-hk-pwm2.dtbo
/boot/dtb-6.12.25-current-rockchip64/rockchip/overlay/rockchip-rk3588-pwm15-m2.dtbo
/boot/dtb-6.12.25-current-rockchip64/rockchip/overlay/rockchip-rk3588-uart4-m2.dtbo
/boot/dtb-6.12.25-current-rockchip64/rockchip/overlay/rockchip-rk3588-pwm2-m1.dtbo
/boot/dtb-6.12.25-current-rockchip64/rockchip/overlay/rockchip-rk3588-i2c8-m2.dtbo
/boot/dtb-6.12.25-current-rockchip64/rockchip/overlay/rockchip-rk3588-pwm3-m2.dtbo
/boot/dtb-6.12.25-current-rockchip64/rockchip/overlay/rockchip-rk3588-pwm1-m2.dtbo
/boot/dtb-6.12.25-current-rockchip64/rockchip/overlay/rockchip-rk3588-pwm5-m2.dtbo
/boot/dtb-6.12.25-current-rockchip64/rockchip/overlay/rockchip-rk3588-pwm13-m2.dtbo
/boot/dtb-6.12.25-current-rockchip64/rockchip/overlay/rockchip-rk3588-pwm14-m2.dtbo
/boot/dtb-6.12.25-current-rockchip64/rockchip/overlay/rockchip-rk3588-uart7-m2.dtbo
/boot/dtb-6.12.25-current-rockchip64/rockchip/overlay/rockchip-rk3588-pwm6-m2.dtbo

dietpiEnv.txt:

root@dietpi:~# cat /boot/dietpiEnv.txt
rootdev=UUID=2a524dbc-55ed-4136-a984-bd87eb4222a5
rootfstype=ext4
# The init system logs to the console defined last.
consoleargs=console=ttyS2,1500000 console=tty1
usbstoragequirks=
extraargs=net.ifnames=0
docker_optimizations=off
overlay_path=rockchip
overlay_prefix=rockchip
overlays=
user_overlays=

[MichaIng]

Thanks for testing. Good to know it is no general issue.

@eggcllnt
I guess this is the essential problem:

[    4.886999] pci 0002:00:00.0: BAR 1 [mem size 0x40000000]: can't assign; no space
[    4.887699] pci 0002:00:00.0: BAR 1 [mem size 0x40000000]: failed to assign

Could you filter your dmesg output to include pci for a successful start as well, to check how it tries to assign the BAR memory in that case?

Seems to be some conflict for assigned resources: https://stackoverflow.com/questions/65241905/linux-pcie-preallocate-bar-space

Do you have 2 GB or 4 GB RAM?

[MichaIng]

Since I was digging into this PCI(e) device tree nodes in another context, here the BAR address range and size defined in the device tree for this PCIe bus: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/arch/arm64/boot/dts/rockchip/rk3568.dtsi?h=linux-6.12.y#n116

This part <0x03000000 0x0 0x40000000 0x3 0x80000000 0x0 0x40000000>;, the first being only the bit field, the second two (64-bit high and low value) the PCI bus address, the next two the CPU bus address to map to, and the last two the size of the segment to map to, which is 1 GiB. Seems like in some boot sessions this is too large 🤔. It is the same for all 3 PCIe busses (2x 2.5 Gbit Ethernet adapters, and 1x M.2 slot) in this last range block, all 1 GiB segment size from the same PCI bus address, but mapped to different CPU bus addresses, with an offset of 1 GiB one after another.

Others can probably debug this faster. But if I shall help, I need the full dmesg output of a positive and a negative boot session, to compare values (in case they differ) and order in which devices have been loaded, to narrow down which could block the needed space on the CPU bus,

If you do not need both 2.5 Gbit Ethernet adapters, we could also disable one and see whether this helps. We could also change the mapping of the M.2 slot to use the CPU bus address previously used by the disabled Ethernet adapter. But I am not sure whether it can simply be done like that, would be experimenting 😄.