EtherREGEN: Installation, Usage, Difficulty, Questions thread

[Dev]

Hi Folks,

 

I received a eR couple of days back and having some network issues for which I need some help

 

My network is as follows:

 

                                                                                     Fiber

Orbi Wireless Router ------> EdgeRouter Switch -------------->  eR (B) ---> NUC7i7 ---> DAC

                                                      ^

                                                       |

                                                    NAS

 

I have reserved IP address for the NUC and NAS in the Orbi. So it supposed to get fixed IP address every time it request for DHCP.

 

In this setup the eR just replaced another switch. Before I got the eR, this network topology have been rock solid as its quiet simple. Every time I rebooted the NUC (or for that matter any devices in the network, like the NAS), it would connect to the network without any glitch. The problem started when I introduced the eR.

 

The issue for me if I reboot the NUC or I disconnect power to the eR and reconnect it back, the NUC would not be able to connect back to the network again. I am running Euphony on the NUC and hence I don't have shell access to troubleshoot anything.

 

On the eR side, I see both the orange (link) and green (activity) leds lit and on the NUC both are green (link is green since it negotiates at 100Mbps). On the Edgerouter, the fiber link is up at 1Gbps.

 

I ran packet capture on the Edgerouter and I don't see any DHCP request coming. Is it possible that eR may not be forwarding it for some reason ?

 

Sometime it works when I reboot the entire network chain - wireless router, edgerouter, eR, NUC and then it would connect but not always, sometime it connects just by rebooting the edgerouter and NUC. So its random and there is something to it that I am not able to understand.

 

I haven't tried setting the link speed specifically at 100Mbps in Euphony as auto-negotiate seems to work but not always - if I disconnect the ethernet cable and reconnect it, some times (not always) the link is negotiated at 10Mbps. Reboot of the NUC fixes this issue though.

 

ethtool eth0
Settings for eth0:
Supported ports: [ TP ]
Supported link modes:   10baseT/Half 10baseT/Full
                       100baseT/Half 100baseT/Full
                       1000baseT/Full
Supported pause frame use: No
Supports auto-negotiation: Yes
Supported FEC modes: Not reported
Advertised link modes:  10baseT/Half 10baseT/Full
                       100baseT/Half 100baseT/Full
                       1000baseT/Full
Advertised pause frame use: No
Advertised auto-negotiation: Yes
Advertised FEC modes: Not reported
Speed: 10Mb/s
Duplex: Full
Port: Twisted Pair
PHYAD: 1
Transceiver: internal
Auto-negotiation: on
MDI-X: on (auto)
Cannot get wake-on-lan settings: Operation not permitted
Current message level: 0x00000007 (7)
      drv probe link
Link detected: yes 

 

Any ideas ? 

 

[Dev]

2 minutes ago, austinpop said:

 

Try turning EEE off on your NUC's NIC? This is just a shot in the dark, given the EEE problems reported in the early firmware.

 

Try ethtool --show-eee eth0 and see what it shows. If EEE is enabled, then:

 

 

ethtool --set-eee eth0 eee off

 

Though EEE is enabled but its inactive on the NUC side since the link partner, i.e. the eR doesn't support it. It also means that the eR has the latest firmware ?

 

I will try to explicitly disabling it but at the moment I can't connect the NUC to the network due to this issue. I am randomly trying different things out but can't make it work. If I still can't connect, I will remove the eR and go from there.

[Dev]

2 minutes ago, austinpop said:

 

You can get shell access by ssh'ing to the Euphony machine using creds audiouser/euphony

 

What you won't have is root access. You should be able to issue the set-eee command using the ethtool line in Advanced settings. Željko told me recently he was working on a way to issue multiple ethtool invocations, so check with him if he has something.

 

Yes, Željko is doing it on my request. I have a ticket filed with him two weeks back.

[Dev]

I think I found the culprit to be eR's SFP port. When I replace the eR with another switch with SFP port, it just works as expected.

 

The Edgerouter end shows the link to be up but it's receiving 0 traffic when its connected to eR using fiber.

 

To troubleshoot this I connected the eR with a long Ethernet cable I had in hand to the Edgerouter (which in another room) and bingo - it started working.

 

Funny that the same SFP worked for couple of days initially. I already had intermittent trouble with connectivity earlier but today it totally stopped working - no matter what I do, it just would work. I tried different transceivers, 850nm, 1310nm but nothing. So my take is that something gone bad with the h/w ?

 

let me know if there is anything else I can try.

 

 

 

 

 

 

 

[Dev]

2 hours ago, JohnSwenson said:

Are all the SFP modules from the  same manufacturer? We have seen tha for a very small number of people that the modules from a company may not fully mate with the connector in the ER. When using modules from different manufacturer the problem goes away.

 

The connector and spacing between the connector and cage all meet the SFP specs, but I'm guessing that once in awhile there are very slight physical differences between modules that can prevent a small number of them from connecting all the pins.

 

We have had reports that people with such a module have been able to fix the problem by very firmly pushing the module into the cage.

 

If this doesn't work changing to a different manufacturer does seem to fix the problem.

 

John S.

 

Hi John,

 

Thanks for the insight. These sfp modules were working fine initially. I tried two different pairs - one from TP Link (SX) and the other is from Planetech (LX) which has been reported to be working fine with eR

 

https://planetechusa.com/product/mgb-tlx-mini-gbic-lx-module-20km/

 

The Edgeroute end shows the link to be up. I am assuming if the connector didn't made contact properly on either end, the link would probably not come up. However, I have no way to check this on the eR end.

 

I will retry pushing it firmly and see if I can make it work.

 

 

[Dev]

1 hour ago, soares said:

I must say that this module didn’t work with my eR. I received on Tuesday and no matter how I pushed it, it doesn’t work. Cheers Jorge

 

 

Can you let us know what worked ? there are number of folks using this transceiver without issues. in fact I got the link from this thread only.

[Dev]

9 minutes ago, Superdad said:

 

Did you buy a matching one for the other end as well as a single-mode fiber cable? Because the transceiver he linked to is a single-mode type.

 

In my case, I am using a pair and single-node fiber off-coarse.

[Dev]

yesterday late night I reset the Edgerouter to factory defaults, updated the firmware (the previous firmware was working fine for the last 5yrs and never had issues) and redid the config (you need to configure it as a managed switch instead of a router), used the TPLink SFP and it started working again. Removed the TPLink and inserted the Planetech on both ends and it still works - so I thought problem solved !

 

This morning disconnected the power from eR, waited 30sec, connected the power and back to the same issue - Edgerouter says the link is up (and its has Tx/Rx power) but no packets received on the fiber interface (eth5).    

 

 

I am still scratching my head as to what else I can debug 🤔

[Dev]

Sorry, more confusion now.....

 

I connected a Linux pc to the A side port and it seems to work just fine. So apparently it doesn't appear to be SFP port issue on either side that I thought earlier. From inside the Linux pc I can't ping the NUC which is connected on the B side. Moving the NUC to the A side and it works just fine.

 

What does that tell ? issue on the B port ? I will do more tests.

[Dev]

1 hour ago, lpost said:

 

I would replace the SFPs and be done with it.  These work great, I prefer 12dB receive attenuator on each end, but they work just fine with a few meter patch.

 

https://www.fs.com/products/12674.html

 

https://www.fs.com/products/48527.html

 

https://www.fs.com/products/68297.html  - 5m is a nice minimum for these SFP made to shoot 10km.  

 

Thanks! With some more tests, looks like the issue is not with the sfp. See the previous post.

[Dev]

2 minutes ago, ASRMichael said:

Just to check, because I had an issue connecting to B side. I has Euphony set to eth0 speed 1000 duplex full autoneg off, I had to change to 100, as B side is only 100mbs. 

 

I have not set anything in Euphony. When it works, I see that the link negotiated to 100Mbps which it should. So I am thinking it should be fine but maybe it worth setting it explicitly.

 

Does eR 'B' port supports auto-negotiation at all ? If it doesn't then that's probably the real issue I am facing....hopefully 🙃

[Dev]

8 minutes ago, Superdad said:

Hi Dev:

The 'B' port of the EtherREGEN is 100Mbps only and it negotiates to that only. If I am not mistaken (@JohnSwenson would correct me), all Ethernet auto-negotiation occurs at 10Mbps before both sides settle to agreed up rate.

 

No, the ethernet does not start "auto-negotiating" at 10Mbps. That's the default speed which both link partner (when the L1 link comes up) starts and then negotiates both the speed and duplex - the lowest of any link partner is the speed and duplex that is finally settled on both sides - all this assumes both the link partner is capable of auto-negotiating.

 

I understand that the eR's B side speed is 100Mbps but does the MAC support auto-negotiation feature ? That's the key question. If it doesn't, then the link partner should set a default speed and duplex and also set auto-negotiation to off. On the other hand if it does, then the partner will automatically negotiate speed to 100Mbps and duplex to full (assuming the eR does as well).