1 Issue:
The Toshiba neutrino ethernet driver cann't ping PC:
Add dts node "qcom,ntn_avb" in our dts, ping success on 100M.
But it ping fail on 1000MHz.
1.1 Test step:
push the Neutrino firmware(DWC_ETH_QOS_fw.bin) to /tmp:
overlay the /lib/firmware/ by tmp
cd /tmp
mkdir upper work
mount -t overlay overlay -olowerdir=/lib,upperdir=/tmp/upper,workdir=/tmp/work /lib
cp /tmp/DWC_ETH_QOS_fw.bin /lib/firmware/
insmod the driver:
扫描二维码关注公众号，回复： 2410781 查看本文章
modprobe DWC_ETH_QOS.ko fw_filename=DWC_ETH_QOS_fw.bin
config device and PC ip on same subnet
ifconfig eth0 192.168.225.3 up
1.2 Test status:
- Use Usb-Ethernet adapter(100M) connect PC, DUT connect the adapter, DUT can ping PC success each other .
- DUT connect PC, ping fail.
- Use 100M SWITCH connect our DUT and PC, DUT can ping PC success each other .
- Use 1000M SWITCH connect our DUT and PC, ping fail.
- change network cable ,the same result.
- use toshiba release driver, the same test result.
2 investigate method
2.1 adjust between 100M and 1000M
from the 100M dmesg log:
DWC_ETH_QOS 0000:01:00.0 eth0: Link is Up - 100Mbps/Full - flow control rx/tx
But from 1000M dmesg log:
DWC_ETH_QOS 0000:01:00.0 eth0: Link is Up - 1Gbps/Full - flow control of
This log from kernel/drivers/net/phy/phy.c phy_print_status() ,and be called by DWC_ETH_QOS_adjust_link.
In DWC_ETH_QOS_adjust_link():
case SPEED_1000:
hw_if->set_gmii_speed(pdata);
hw_if->ntn_set_tx_clk_125MHz(pdata);
break;
case SPEED_100:
hw_if->set_mii_speed_100(pdata);
hw_if→ntn_set_tx_clk_25MHz(pdata);
Eth0 related attribute node:
root@swi-mdm9x40:~# cd /sys/class/net/eth0
root@swi-mdm9x40:/sys/devices/80000.qcom,pcie/pci0000:00/0000:00:00.0/0000:01:00.0/net/eth0# ls
addr_assign_type device mtu statistics
addr_len dormant name_assign_type subsystem
address duplex netdev_group tx_queue_len
broadcast flags operstate type
carrier ifalias phys_port_id uevent
carrier_changes ifindex power
dev_id iflink queues
dev_port link_mode speed
Read speed node ,the speed value is correct.
From dmesg log and speed , we know adjust function is normal.
2.2 tcpdump
100M tcpdump :
root@swi-mdm9x40:/tmp# tcpdump -i eth0 -v
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
00:58:48.551237 IP (tos 0x0, ttl 128, id 31617, offset 0, flags [none], proto UDP (17), length 96)
192.168.225.2.netbios-ns > 192.168.225.255.netbios-ns: NBT UDP PACKET(137): REGISTRATION; REQUEST; BROADCAST
00:58:49.301272 IP (tos 0x0, ttl 128, id 31618, offset 0, flags [none], proto UDP (17), length 96)
192.168.225.2.netbios-ns > 192.168.225.255.netbios-ns: NBT UDP PACKET(137): REGISTRATION; REQUEST; BROADCAST
00:58:50.802455 IP (tos 0x0, ttl 128, id 31620, offset 0, flags [none], proto UDP (17), length 96)
192.168.225.2.netbios-ns > 192.168.225.255.netbios-ns: NBT UDP PACKET(137): REGISTRATION; REQUEST; BROADCAST
00:58:51.553519 IP (tos 0x0, ttl 128, id 31621, offset 0, flags [none], proto UDP (17), length 214)
192.168.225.2.netbios-dgm > 192.168.225.255.netbios-dgm: NBT UDP PACKET(138)
00:58:51.553755 IP (tos 0x0, ttl 128, id 31622, offset 0, flags [none], proto UDP (17), length 214)
192.168.225.2.netbios-dgm > 192.168.225.255.netbios-dgm: NBT UDP PACKET(138)
00:58:51.553894 IP (tos 0x0, ttl 128, id 31623, offset 0, flags [none], proto UDP (17), length 244)
192.168.225.2.netbios-dgm > 192.168.225.255.netbios-dgm: NBT UDP PACKET(138)
00:58:51.760568 IP (tos 0x0, ttl 128, id 31624, offset 0, flags [none], proto UDP (17), length 78)
192.168.225.2.netbios-ns > 192.168.225.255.netbios-ns: NBT UDP PACKET(137): QUERY; REQUEST; BROADCAST
00:58:52.509672 IP (tos 0x0, ttl 128, id 31625, offset 0, flags [none], proto UDP (17), length 78)
192.168.225.2.netbios-ns > 192.168.225.255.netbios-ns: NBT UDP PACKET(137): QUERY; REQUEST; BROADCAST
00:58:53.260730 IP (tos 0x0, ttl 128, id 31626, offset 0, flags [none], proto UDP (17), length 78)
192.168.225.2.netbios-ns > 192.168.225.255.netbios-ns: NBT UDP PACKET(137): QUERY; REQUEST; BROADCAST
00:58:55.629145 IP (tos 0x0, ttl 128, id 31627, offset 0, flags [none], proto UDP (17), length 229)
192.168.225.2.netbios-dgm > 192.168.225.255.netbios-dgm: NBT UDP PACKET(138)
00:58:57.453302 IP6 (hlim 1, next-header UDP (17) payload length: 125) fe80::688b:67ec:b117:4bc6.dhcpv6-client > ff02::1:2.dhcpv6-server: [udp sum ok] dhcp6 solicit (xid=aeeb81 (elapsed-time 3100) (client-ID hwaddr/time type 1 time 564570276 4437e69c01e4) (IA_NA IAID:50335430 T1:0 T2:0) (Client-FQDN) (vendor-class) (option-request vendor-specific-info DNS-server DNS-search-list Client-FQDN))
00:58:57.871096 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.225.3 tell 192.168.225.2, length 46
00:58:57.871269 ARP, Ethernet (len 6), IPv4 (len 4), Reply 192.168.225.3 is-at 0a:a1:f6:cf:85:37 (oui Unknown), length 28
00:58:57.873740 IP (tos 0x0, ttl 128, id 331, offset 0, flags [none], proto ICMP (1), length 60)
192.168.225.2 > 192.168.225.3: ICMP echo request, id 1, seq 461, length 40
00:58:57.874224 IP (tos 0x0, ttl 64, id 53005, offset 0, flags [none], proto ICMP (1), length 60)
192.168.225.3 > 192.168.225.2: ICMP echo reply, id 1, seq 461, length 40
00:58:58.3
1000M tcpdump:
root@swi-mdm9x40:/tmp# tcpdump -i eth0 -v
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
00:02:34.452777 IP (tos 0x0, ttl 128, id 31240, offset 0, flags [none], proto UDP (17), length 96)
192.168.225.6.netbios-ns > 192.168.225.255.netbios-ns: NBT UDP PACKET(137): REGISTRATION; REQUEST; BROADCAST
00:02:35.205422 IP (tos 0x0, ttl 128, id 31241, offset 0, flags [none], proto UDP (17), length 96)
192.168.225.6.netbios-ns > 192.168.225.255.netbios-ns: NBT UDP PACKET(137): REGISTRATION; REQUEST; BROADCAST
00:02:36.705548 IP (tos 0x0, ttl 128, id 31243, offset 0, flags [none], proto UDP (17), length 96)
192.168.225.6.netbios-ns > 192.168.225.255.netbios-ns: NBT UDP PACKET(137): REGISTRATION; REQUEST; BROADCAST
00:02:37.455928 IP (tos 0x0, ttl 128, id 31244, offset 0, flags [none], proto UDP (17), length 96)
192.168.225.6.netbios-ns > 192.168.225.255.netbios-ns: NBT UDP PACKET(137): REGISTRATION; REQUEST; BROADCAST
00:02:38.206693 IP (tos 0x0, ttl 128, id 31245, offset 0, flags [none], proto UDP (17), length 96)
192.168.225.6.netbios-ns > 192.168.225.255.netbios-ns: NBT UDP PACKET(137): REGISTRATION; REQUEST; BROADCAST
00:02:38.956778 IP (tos 0x0, ttl 128, id 31246, offset 0, flags [none], proto UDP (17), length 214)
192.168.225.6.netbios-dgm > 192.168.225.255.netbios-dgm: NBT UDP PACKET(138)
00:02:38.957017 IP (tos 0x0, ttl 128, id 31247, offset 0, flags [none], proto UDP (17), length 214)
192.168.225.6.netbios-dgm > 192.168.225.255.netbios-dgm: NBT UDP PACKET(138)
00:02:38.957153 IP (tos 0x0, ttl 128, id 31248, offset 0, flags [none], proto UDP (17), length 244)
192.168.225.6.netbios-dgm > 192.168.225.255.netbios-dgm: NBT UDP PACKET(138)
00:02:44.429021 IP6 (hlim 1, next-header UDP (17) payload length: 125) fe80::e119:293a:961d:9466.dhcpv6-client > ff02::1:2.dhcpv6-server: [udp sum ok] dhcp6 solicit (xid=c61a45 (elapsed-time 3100) (client-ID hwaddr/time type 1 time 564570276 4437e69c01e4) (IA_NA IAID:98093962 T1:0 T2:0) (Client-FQDN) (vendor-class) (option-request vendor-specific-info DNS-server DNS-search-list Client-FQDN))
00:02:45.734636 IP (tos 0xc0, ttl 1, id 0, offset 0, flags [DF], proto IGMP (2), length 32, options (RA))
0.0.0.0 > 224.0.0.1: igmp query v2
00:02:49.689747 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.225.3 tell 192.168.225.6, length 46
00:02:49.689857 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.225.6 tell 192.168.225.1, length 28
00:02:49.689894 ARP, Ethernet (len 6), IPv4 (len 4), Reply 192.168.225.3 is-at 8e:4c:c0:c7:da:12 (oui Unknown), length 28
00:02:50.369120 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.225.3 tell 192.168.225.6, length 46
00:02:50.369304 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.225.6 tell 192.168.225.1, length 28
00:02:50.369452 ARP, Ethernet (len 6), IPv4 (len 4), Reply 192.168.225.3 is-at 8e:4c:c0:c7:da:12 (oui Unknown), length 28
00:02:51.368220 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.225.3 tell 192.168.225.6, length 46
00:02:51.368402 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.225.6 tell 192.168.225.1, length 28
00:02:51.368549 ARP, Ethernet (len 6), IPv4 (len 4), Reply 192.168.225.3 is-at 8e:4c:c0:c7:da:12 (oui Unknown), length 28
00:02:52.377134 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.225.3 tell 192.168.225.6, length 46
00:02:52.377314 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.225.6 tell 192.168.225.1, length 28
00:02:52.377464 ARP, Ethernet (len 6), IPv4 (len 4), Reply 192.168.225.3 is-at 8e:4c:c0:c7:da:12 (oui Unknown), length 28
00:02:53.367975 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.225.3 tell 192.168.225.6, length 46
00:02:53.3681
From tcpdump , we know rx can receive packet ,but tx transfer data fail.
2.3 discuss with other colleague
Discuss with other colleague.
The most import message is Rx/Tx clock internal delay needs to be set for interfacing PHY in RGMII mode.
2.4 PHY register
1000M :
[ 59.528722] ************* PHY Reg dump *************************
[ 59.572582] Phy Control Reg(Basic Mode Control Reg) (0x0) = 0x0
[ 59.582683] Phy Status Reg(Basic Mode Status Reg) (0x1) = 0x7949
[ 59.590262] Phy Id (PHYS ID 1) (0x2)= 0x4d
[ 59.596441] Phy Id (PHYS ID 2) (0x3)= 0xd074
[ 59.602585] Auto-nego Adv (Advertisement Control Reg) (0x4) = 0x1de1
[ 59.608440] net eth0: PHY already attached
[ 59.613481] Auto-nego Lap (Link Partner Ability Reg) (0x5)= 0x0
[ 59.620745] Auto-nego Exp (Extension Reg)(0x6) = 0x0
[ 59.632053] Auto-nego Np (0x7) = 0x0
[ 59.636903] IPA uC is ready -17
[ 59.639178] Extended Status Reg (0xf) = 0xa000
[ 59.644928] ipa ipa2_setup_uc_ntn_pipes:345 client 15 (ep: 12) connected
[ 59.651249] ipa ipa2_setup_uc_ntn_pipes:377 client 50 (ep: 1) connected
[ 59.657230] 1000 Ctl Reg (1000BASE-T Control Reg)(0x9) = 0x200
[ 59.666409] -->DWC_ETH_QOS_vlan_rx_add_vid: vid = 0
[ 59.670659] <--DWC_ETH_QOS_vlan_rx_add_vid
[ 59.675556] 1000 Sts Reg (1000BASE-T Status)(0xa) = 0x0
[ 59.686803] PHY Ctl Reg (0x10) = 0x862
[ 59.692800] PHY Sts Reg (0x11) = 0x10
[ 59.695433]
[ 59.695433] ****************************************************
100M:
[ 62.618699] ************* PHY Reg dump *************************
[ 62.662587] Phy Control Reg(Basic Mode Control Reg) (0x0) = 0x0
[ 62.672355] Phy Status Reg(Basic Mode Status Reg) (0x1) = 0x7949
[ 62.679800] Phy Id (PHYS ID 1) (0x2)= 0x4d
[ 62.687296] Phy Id (PHYS ID 2) (0x3)= 0xd074
[ 62.692517] net eth0: PHY already attached
[ 62.695634] Auto-nego Adv (Advertisement Control Reg) (0x4) = 0x1de1
[ 62.705455] Auto-nego Lap (Link Partner Ability Reg) (0x5)= 0x0
[ 62.717345] Auto-nego Exp (Extension Reg)(0x6) = 0x6
[ 62.724898] Auto-nego Np (0x7) = 0x1000
[ 62.727737] IPA uC is ready -17
[ 62.732435] ipa ipa2_setup_uc_ntn_pipes:345 client 15 (ep: 12) connected
[ 62.738140] Extended Status Reg (0xf) = 0xa000
[ 62.743426] ipa ipa2_setup_uc_ntn_pipes:377 client 50 (ep: 1) connected
[ 62.751090] 1000 Ctl Reg (1000BASE-T Control Reg)(0x9) = 0x200
[ 62.761451] -->DWC_ETH_QOS_vlan_rx_add_vid: vid = 0
[ 62.765500] <--DWC_ETH_QOS_vlan_rx_add_vid
[ 62.771067] 1000 Sts Reg (1000BASE-T Status)(0xa) = 0x0
[ 62.785527] PHY Ctl Reg (0x10) = 0x862
[ 62.792702] PHY Sts Reg (0x11) = 0x10
[ 62.795335]
[ 62.795335] ****************************************************
From the PHY Reg dump, we have't found obviouse diffrent between 100M and 1000M.
3 solution
Rx/Tx clock internal delay needs to be set for interfacing PHY in RGMII mode.As per PHY spec, RX has the delay by default ,while we need to enable this delay for TX.
Neutrino does not support internal delay for RGMII Rx/Tx clock lines,these delays need to be enabled on PHY or implemented on the board.
For AR8033, debug registers defined by Qualcomm Atheros:
Write the offset address of debug register to 0x1D.
Read/write the data from/to 0x1E.
We need operate SerDes test and system mode control register,the offset is 0x05:
0x1D and 0x1E is debug port which 0x1D is address offset register and 0x1E is dataport register.
We write offset(0x05) to 0x1D:
DWC_ETH_QOS_mdio_write_direct(pdata, pdata->phyaddr, 0x1D, phyreg);
Set 0x05 bit 8 to 1:
DWC_ETH_QOS_mdio_read_direct(pdata, pdata->phyaddr, 0x1E,&phydata);
phydata |= 0x1<<8;
Write phydata to 0x1E:
DWC_ETH_QOS_mdio_write_direct(pdata, pdata->phyaddr, 0x1E, phydata);
After we resolved ping fail issue ,we find a packet loss issue in 1000M.
Pinging 192.168.225.3 with 32 bytes of data:
Reply from 192.168.225.3: bytes=32 time=4ms TTL=64
Reply from 192.168.225.3: bytes=32 time=2ms TTL=64
Request timed out.
Reply from 192.168.225.3: bytes=32 time=3ms TTL=64
Use 1000M SWITCH connect our DUT and PC, ping success and has't packet loss.
Disable EEE feature in PC , ping is successful.
Then I found it is EEE(Energy Efficient Ethernet) 802.3az cause the packet loss issue and comment out DWC_ETH_QOS_eee_init() can resloved it .
From https://en.wikipedia.org/wiki/Energy-Efficient_Ethernet:
Energy-Efficient Ethernet (EEE) is a set of enhancements to the twisted-pair and backplane Ethernet family of computer networking standards that
allow for less power consumption during periods of low data activity.
The Institute of Electrical and Electronics Engineers (IEEE), through the IEEE 802.3az task force developed the standard.
The IEEE ratified the final standard in September 2010.
802.3az is a new standard that maybe cause incompatible between different company hardware
And it is applicable that disable EEE feature in our project by comment out DWC_ETH_QOS_eee_init().
4 summary
And we read driver code but found we haven't enough message in TC9550 datasheet.
Discuss meet is very import for block issue.
Hardware company maybe have rich experience about their hardware driver.
Search similar issue in network can provide a important thinking for us.