Optical Network Configurations

[plissken]

5 hours ago, R1200CL said:

@plissken

 

I guess you’re the one to ask. 
These modules is half price compared to the original FTLX1475D3BTL

 

https://www.gbic-shop.de/FTLX1475D3BTL-kompatibel

Should I expect such modules to comply 100% with the original Finisar data sheet ?

 

That would be a question for the OEM. Looks like that Finisar module is $105 for 10GB LR. All I know is we almost exclusively use FS.com modules. We still have some customers the have a hard requirement for vendor specific but that's now an outlier.

[plissken]

Always check the logging in your managed switch when working these sort of issues.

[plissken]

22 minutes ago, Duke40 said:

I'll stick with the 1Gb SFPs like Finisar1321 and Planet Techs, they work a treat.

 

This is a good recommendation. Just get the SX LC (850nm Multimode) 1GBe transceivers. They are like $6-$8 and the cabling is equally affordable. At 125MB/s they are 12500% more bandwidth than you need for even 24/192 PCM. You get all your ground plane isolation that you could hope for.

[plissken]

11 minutes ago, jabbr said:

The reason to use 10Gbe Ethernet e.g. 10Gbase-X is that it measurably obliterates any upstream effect of clocks. External clocks are pointless in home networks. If you are interested in clock synronization then check out the Solarflare Flareon Ultra e.g. https://discover.exertisenterprise.com/solarflare/assets/files/Solarflare_Precision_Time_Product_Brief.pdf

These NICs exceed the 10Gbe requirements that said I've compared these to 100Gbe Ethernet (which has even more extreme clock requirements e.g. Mellanox ConnectX-4,5 and frankly can't hear a shred of difference over a regular fiberoptic ethernet connection.  

 

 

I don't know that standard but on initial read you have to have an endpoint that also supports PTP/IEEE 1588(?) Otherwise it's a non-feature.

[plissken]

1 hour ago, Bertel said:

or introduce new jitter on its own at 1GbE? Just trying to understand and be sure ;-)

 

No.  This is not the way Ethernet works. Look at any Ethernet PHY block diagram and you will see independent, high speed, xmit and xrec buffers. They rewrite the frame and regenerate the signal going back out on their Tx segment. If you have:

 

Router<>Switch MDF<>SwithchIDF_1<>SwitchIDF_2<>end point and your router would have a server plugged into it your data stream down to you would be rewritten 4 times.

 

Why anyone is placing an uber expensive switch at the router portion is beyond me.

 

Get a Mikrotik/Ubiquiti/HPE Aruba etc and SFP's from an outfit like FS.COM and keep it fiber all the way to your copper hand off.

[plissken]

5 hours ago, Bertel said:

So you're saying that the capability to not transmit upstream jitter lies in the hardware design, lauout and build of the switch, not in the protocol or transmission speed of the 10+GbE connectivity itself?

 

Modern, and most default, switching is called Store and Forward. It stores the incoming frame and regenerates the frame from scratch and then forwards it on.

 

Jitter as far as Layer 1 only has to do with the transmitter and receiver that are talking and only for that one point to point link.

 

For all you know your HD stream from Tidal, Amazon, or Apple, could be going over multiple different connection types all with their own point to point signal integrity requirements. None of that matters to us because we receive the data Asynch. It's a bit bandwidth intensive and latency insensitive.

 

I teach the CCNA, I'm a dual CCNP and ACEP. I do VOIP, I do medical emergency paging systems, I do casino Sec_Cam's, I do broadcast so live A/V and multi-cast IGMP/PIM are all in my wheelhouse. I just worked with a casino on a 300 camera project that cover all the tables in addition to causeways, entrances, cashiers etc...

[plissken]

1 hour ago, barrows said:

 A switch such as the Microtik referenced in this thread does not contain a clock with nearly the performance as what is internal to the oR and Signature Rendu SEoptical.

 

How does this manifest past the interface I/O buffer where we have internal ASIC handoff or ASIC to ASIC handoff over some internal backplane?

[plissken]

31 minutes ago, jabbr said:

Agreed that there is zero evidence that an external clock of any type does anything to improve the functionality of a network switch let alone the SQ of audio reconstructed from asynchronous packets transmitted through the switch. For a variety of engineering reasons using external clocks is pointless.

 

I've yet to see any switch from any big iron vendor have an external clock input. Doesn't matter if it's Cisco, Juniper, Arista, HPE Aruba, Extreme, Huawei, Brocade etc...

[plissken]

1 minute ago, The Computer Audiophile said:

I believe they use switches with external clocks at CERN. 
 

https://white-rabbit.web.cern.ch

 

You have a CDW or Synnex SKU # for this commercial offering?

[plissken]

I did broadcast for 7 years. You need house clock for this stuff. AVB, AES67 protocols used on switches are no different. You have to have incredibly accurate timing to avoid sync issues like frame tear.

 

 

[plissken]

4 hours ago, Bertel said:

Which interestingly brings back the question whether a version of the opticalRendu that would be able to receive and sync to an external time sync information like PPS or CLK from the WRS-LJ (or an external source when both the switch and the Rendu sync to it) would make sense and settle the aspect of jitter over 1GbE...

 

Bottom line and it's an easy bottom line: External house sync only matters if you are dealing with realtime data. Playback of prerecorded A/V already has all it's clocking embedded. The switch/router is going to simply pass this along at it's best effort, with regards to config, and the final end point will extract that clocking along with the A/V and play it back out.

 

 

[plissken]

15 hours ago, Jud said:

Dumb question: Will an SFP module fit and work correctly in an SFP+ slot?

 

By spec SFP+ is supposed to be backwards compatible with SFP. I use 1/10gbe transceivers interchangeably all the time.

[plissken]

On 8/21/2021 at 3:27 PM, Jud said:

Mesh wi-fi routers coordinate with each other using a "backhaul." 

 

Mesh wifi are an AP that have enough CPU power to elect one as a Wireless LAN controller. They could also have a specialty built unit that have additional WAN port for L3 to your ISP. That would be a router with a WAP built in.

 

"Backhaul" is another term for the management vlan that they send any SA, SE, stats, beacon, and other types of traffic. It carries no data traffic. Can be totally L2 frames and most likely is.

 

Routing has the connotation that traffic is leaving a broadcast domain and making L3 hops to another destination.

[plissken]

3 hours ago, The Computer Audiophile said:

That's not how I've traditionally seen the term used for WiFi home networks. I'm not saying your incorrect,

 

I didn't say backhaul is an incorrect term.

 

Now traditionally speaking:

 

Backhaul is taking traffic that sources at an AP or Edge switch and hauling it all the way back to core routing to determine where we egress it. I just finished setup of a school where everything from AP to edge switch is an encrypted tunnel back to the controllers. If this was a large multi-site campus we would put in inexpensive consumer circuits locally at the site and have a local hand off for all undefined traffic so it doesn't traverse our MPLS or our dedicated fiber links.

 

[plissken]

20 hours ago, hlkaye said:

For an unskilled computer user hesitant to use a managed 10G switch like the much recommended Mikrotik CRS305, might this QNAP unmanaged switch (QSW-2104-2S  https://www.qnap.com/en-us/product/qsw-2104-2s) be an acceptable alternative, or is the use of regular RJ45 ports rather than the SFP RJ45 a problem.  would this compromise the benefits of using a 10G switch? I would use the 2 10Gbe SFP+ plus ports to connect my SGC i5 server and a NAS.  My work PC would be connected to the switch via one of the ethernet ports.

Or is the set up of the Mikrotik switch easy enough even for a novice like me and worth the little bit of effort?

 

If you want an easier to manage Switch look at the Aruba Instant On series. You can get a 24 port copper / 4 SFP/SFP+ for ~$250. You can use it as a dumb switch and then walk it toward more management rich features if you wish.

 

 

 

 

[plissken]

18 hours ago, hlkaye said:

Thank you for the advice.  That's very helpful.

May I ask what "Full Duplex" and "Flow Control" are, and why you have selected them and "tx only"? 

 

Flow control is a congestion management mechanism. When a link starts getting saturated the switch signals for the end points to start initiating a back off routine to lessen the amount of data they are sending.

 

There are some 'specialty' audiophile network devices that for some reason don't have their own avoidance mechanisms built in (which they should) they require this ancient standard. Tx means we only look at transmitter of traffic and back them off.

 

I architect and engineer networks for a living (currently working on a 4 campus, 3 data center, 137 switches, 700 wireless access points) and there will be zero Flow Control configs.

[plissken]

10 minutes ago, ericuco said:

So is it your advise to turn off flow control, assuming no “specialty” audiophile network devices are being used, as in my case?

 

My latest Mikrotik switch had flow control turned off by default.

 

Hopefully control mechanisms like Flow Control, policing, shaping, CoS, QoS, will come as a best practices paper from the manufacturer.

 

Per Cisco guidelines flow control shouldn't be used because it's application unaware and mechanisms to handle that should be up layer in the OSI stack. That is software devs should program properly. But with that said if you have a purveyor of oddity audiophile networking gear and they suggest flow control and you want to use it then do so. My issue is that it signals that the vendor may not be as network savvy as they should be if that's how they've chosen to solve their 'issues'.

 

Flow control is a 'trick' to send a back off frame to an endpoint. An endpoint that also may be sending out other critical data unrelated to the data that caused the congestion.

[plissken]

5 minutes ago, ericuco said:

I did notice that when flow control was turned on, for some ports a Tx only option (check box) would show up for some but not others. Does that mean that the connected equipment has the proper controls in place?

 

That would be most likely a vendor option. Are the ports it's not showing up for half duplex?

Quote

Also, my latest Mikrotik switch is running SwOS Lite. One of things it appears to monitor is the connections for each port to see if there is an issue along the cables. For instance, my NUC kept switching to 100M speed (vs 1 Gb) for some reason. Under the Cable Pair column, a "symbol" would show up. When I researched it, this meant that something was wrong in the cabling. Supposedly, the "symbol" was to indicate where along the cable the issue occurred.  I switched  cables and the issue went away (connection stays at 1 Gb). I was using a rather short (0.5M) cable which might be the issue and am now using a longer (2M) cable. In any case, it was very handy to 1) realize a problem existed (connecting at 100M vs 1G) and 2) what was causing the problem.

 

I haven't dug into the Mikrotik at all but I know other switch manufactures have an active cable diagnostic that can give you the length of the cable and any open/shorts. Pretty handy.

 

Another thing to check on is per port counters and on something that was flapping you between 1Gbe and 100Mbe see if there were excessive CRC alignment or FCS errors.

[plissken]

1 hour ago, StreamFidelity said:

I use an Afterdark Switch: Buffalo BS-GS2016 SFP+ 10G Cascade & Giesemann OCXO and first had to activate flow control to connect the Solarflare X2522 NIC. 

 

I've installed THOUSANDS of 10Gbe and up interfaces and yet to enable flow control to get a link up/up.

 

I'm standing pat with Cisco on this:

 

Flow control is a last resort when you have a vendor that isn't managing traffic up layer. It's a ham-fisted one size fits all solution.

 

Take this case in point: You have a device that serves a few functions including VoiP which would also include E911. You have another app on that device that is a piece of shit app and it doesn't manage it's own data flows. It saturates a link and the only way to deal with the congestion is to enable some ancient, dumb, mechanism on the switch to force the Tx'ing NIC to back the fuck off.

 

Now lets say you need to make an E911 call but you can't get DSCP 24 call control traffic to get you a dial tone because flow control is busy backing off the NIC because of some other poorly thought out application.

[plissken]

1 hour ago, StreamFidelity said:

I think ethernet is pretty complicated and works differently for everyone.

 

Ethernet can be pretty complicated and that's why small boutique manufacturers can't get it correct. It's not in their wheelhouse.

[plissken]

18 minutes ago, The Computer Audiophile said:

 

 

Let's just use real world audio examples here. We have many very low power audio endpoints that will get overrun if even a 1Gbe server sends them data at full speed. It seems like flow control is the best option in this scenario. 

 

 

 

From a Cisco blog in 2015:

 

"The general idea is to let the flow control be managed higher up the stack in the form of congestion control. This can be done by applications, and honestly, should be done by the applications as hardware flow control is not application aware"

 

I did say if your vendor, normally boutique dabbling in the deep end of the networking pool, says to enable flow control, then enable it. It means they are never going to bother putting the effort to manage this up stack.

 

It's also a vendor I'll never purchased from. Just like if you create a streaming product that is constantly buffering. I'm not buying that either.

[plissken]

11 minutes ago, The Computer Audiophile said:

Isn't that just an opinion on what Cisco prefers? If some of us want an audio endpoint to do as little as possible, it seems like relying on external hardware to be the traffic cop is a good thing. 

 

I only work with a bunch of CCIE's, Python Devs, etc.... That's just their opinion too. You place your bets and make your purchases. All I know is I have no flow control enabled for my Pi4 wireless streamers running Ropieee XL and even with 24/192 PCM zero issues and checking PTRG shows 0% packet loss and 0% re-transmits on my lowly $56 TP-Link AP's, over my $60 Netgear GSM 7352v2 ebay switch with my $300 Dell R620 server with 10GBE Fiber that also run's pFSense since it's a 4 port NIC in the server (two fiber, two copper).

 

All network gear for the cost of one slow, un-managed, hot, zero efficacy, 6 port switch. *$528 actually. So less than.

[plissken]

7 minutes ago, The Computer Audiophile said:

Again, we should stick in the real world of audio devices. 

 

I have low power audio endpoints that choke with very high data rates from a server. 

 

If I use a crappy network with $56 TP Link APs, that will cause enough slow down to make the endpoints work just fine. Wireless always makes it work because it's slow and has higher latency. 

 

When I connect everything wired without flow control I break it. Enable flow control, fix it. 

 

I'm not talking about any "audio" switch, only audio endpoints that do nothing but audio. CCIEs, Python devs, should really look at specific use cases before making statements. Eat sleep and breathe this stuff for a while, then make an educated statement.

 

Chris, I built my first set of speakers, Karlson 1/4 wave folded horns in 1987.

 

CCIE's and Devs would look at solutions and when they see that the only congestion management available on a particular piece of gear is flow control they would keep on looking.  Is that specific use case enough?

 

My crappy $60 switch with multiple $56 1350AC AP's that I get 38MB/s over with no need of technologies from 1994 to make operate? I'll take that all day long. I'll let our relative networking and audio design and engineering abilities speak for themselves.

[plissken]

21 minutes ago, ericuco said:

Since I have replaced the opticalRendu with the fitlet2 (w/ SFP port) several months ago, I have not had one drop out (knock on wood). So I can only conclude that there was issues with the opticalRendu.

 

This^^. Just a few times in the past I've been contacted by PM to help. My suggestion in those instances was to simply return the oR.  Didn't matter what switch vendor we ended up using it was never going to work properly.

 

 

[plissken]

3 minutes ago, The Computer Audiophile said:

Two very different devices. One has a max network capability of 400Mbps. 

 

Well I was just helping those members, and I quote, "should really look at specific use cases ".

 

We did and determined better, more cost effective, solutions that now work.