Optical Network Configurations

[The Computer Audiophile]

I think I would get this switch if I knew it would work well with AES67 / Ravenna. The Mikrotik forum seems to show some issues or at least unanswered questions. 

[The Computer Audiophile]

The Ravenna recommendations are here - https://www.ravenna-network.com/wp-content/uploads/2022/01/RAVENNA-Network-Requirements-Draft-0.3.pdf

[jabbr]

I'm guessing this hardware is new enough that no one can specifically say they've used it w Ravenna but also no reason to think it wouldn't... OTOH 100Gbe can easily overflow the computer's buffers so you need to allocatate prob both a larger queue as well as larger buffers

[The Computer Audiophile]

I wonder if this list is up to date. 
 

 

PTPv2

Device support

Supported on:
CRS326-24G-2S+ supported only on Gigabit Ethernet ports
CRS328-24P-4S+ supported only on Gigabit Ethernet ports
CRS317-1G-16S+ supported on all ports
CRS326-24S+2Q+ supported on SFP+ and QSFP+ interfaces
CRS312-4C+8XG supported on all ports
CRS318-16P-2S+ supported only on Gigabit Ethernet ports

Not supported on:
CRS305-1G-4S+
CRS309-1G-8S+
CRS328-4C-20S-4S+
CRS354-48G-4S+2Q+

 

https://help.mikrotik.com/docs/display/ROS/Precision+Time+Protocol

[jabbr]

42 minutes ago, The Computer Audiophile said:

I wonder if this list is up to date. 
 

 

PTPv2

Device support

Supported on:
CRS326-24G-2S+ supported only on Gigabit Ethernet ports
CRS328-24P-4S+ supported only on Gigabit Ethernet ports
CRS317-1G-16S+ supported on all ports
CRS326-24S+2Q+ supported on SFP+ and QSFP+ interfaces
CRS312-4C+8XG supported on all ports
CRS318-16P-2S+ supported only on Gigabit Ethernet ports

Not supported on:
CRS305-1G-4S+
CRS309-1G-8S+
CRS328-4C-20S-4S+
CRS354-48G-4S+2Q+

 

https://help.mikrotik.com/docs/display/ROS/Precision+Time+Protocol

I don't see support for PTP ... then again I know nothing about this!

[juniorbudel]

On 1/16/2023 at 6:57 AM, dbastin said:

If you change a cable or power supply on your dirty side, can you hear any difference?

I bought a linear power supply to do this experiment, I'll get back with the reports as soon as it arrives.

[juniorbudel]

On 16/01/2023 at 22:20, dbastin said:

Que tal comparar cabos UTP com cabos STP.

I hadn't thought of that possibility, has anyone reported differences between shielded or unshielded network cable on the “dirty” side of the system?

[barrows]

12 minutes ago, juniorbudel said:

I hadn't thought of that possibility, has anyone reported differences between shielded or unshielded network cable on the “dirty” side of the system?

I put an "audiophile" Ethernet cable from my switch to my opticalModuledeluxe, and noticed an improvement.  The cable was from Sablon, and this Ethernet cable is designed to reduce noise.  I can only surmise that the noise reduction must have allowed the Ethernet clock in the oMd to operate with less phase noise...  I have not tried other "audiophile" Ethernet cables, but I can recommend trying a Sablon for your most important link, note it is a beast of cable, thick and heavy-but also note it does not create a ground connection from end to end, so no worries with it in terms of creating ground loops (many large, shielded, Ethernet cables tie the shields to the plug shells at each end, creating the possibility of causing ground loop currents/noise).  I would recommend avoiding any shielded Ethernet cables which tie the shields to the plug shells at each end, this is just asking for trouble.  If one tests these types of cables, the shield itself might not be causing the "difference" it might just be that one adds a ground loop, and ground loop noise.

[audiobomber]

50 minutes ago, juniorbudel said:

I hadn't thought of that possibility, has anyone reported differences between shielded or unshielded network cable on the “dirty” side of the system?

 

50 minutes ago, juniorbudel said:

I hadn't thought of that possibility, has anyone reported differences between shielded or unshielded network cable on the “dirty” side of the system?

Yes, the ethernet cable choice matters, regardless of where it is located in the chain, IME. 

 

I've been trying various cables from my NAS to a switch and FMC, and the sonic changes are obvious.

[jabbr]

There are a bunch of Intel 100Gbase-CWDM4 (single mode) QSFP28 modules available on eBay at crazy good prices...

[jabbr]

1 hour ago, audiobomber said:

 

Yes, the ethernet cable choice matters, regardless of where it is located in the chain, IME. 

 

I've been trying various cables from my NAS to a switch and FMC, and the sonic changes are obvious.

 

That suggests a serious fault in your FMC/switch, either its 1Gbe and not tested, or if its 10Gbe+ then its not compliant with testing. 10Gbe+ compliance testing does not allow upstream noise/jitter to be propagated by a complaint device.

[barrows]

10 minutes ago, jabbr said:

 

That suggests a serious fault in your FMC/switch, either its 1Gbe and not tested, or if its 10Gbe+ then its not compliant with testing. 10Gbe+ compliance testing does not allow upstream noise/jitter to be propagated by a complaint device.

Not if one ascribes to John Swenson's theory of Ethernet clock phase noise propagating noise in the downstream devices even in the case of optical Ethernet distribution.

[jabbr]

2 hours ago, barrows said:

I put an "audiophile" Ethernet cable from my switch to my opticalModuledeluxe, and noticed an improvement.  The cable was from Sablon, and this Ethernet cable is designed to reduce noise.  I can only surmise that the noise reduction must have allowed the Ethernet clock in the oMd to operate with less phase noise...

 

I know its not required for 1Gbe but have you tested the device with the "stressed receiver test"? Its documented in the 10Gbe standard

 

This article from 2003 gives a fairly English language description and pointers to the actual standards:

https://www.lightwaveonline.com/optical-tech/article/16649441/10gigabit-ethernet-devices-stressed-by-new-test

 

https://download.tek.com/document/65W_26048_0_Letter_0.pdf

https://www.researchgate.net/figure/An-Eye-diagram-that-is-being-stressed-by-jitter_fig2_252990697

[jabbr]

2 minutes ago, barrows said:

Not if one ascribes to John Swenson's theory of Ethernet clock phase noise propagating noise in the downstream devices even in the case of optical Ethernet distribution.

That was specifically considered in the 1990s and ensured against in the millenial era 10 Gbe standard ancient history. Since you know that this might occur, why wouldn't he have designed the equipment not to propagate jitter and other types of noise?

[audiobomber]

1 hour ago, jabbr said:

 

That suggests a serious fault in your FMC/switch, either its 1Gbe and not tested, or if its 10Gbe+ then its not compliant with testing. 10Gbe+ compliance testing does not allow upstream noise/jitter to be propagated by a complaint device.

Everything here is 1Gbe. The switch is a Silent Angel N8, with linear power supply. The FMC is a TP-Link MC220L, SFP's are Finisar 1318 with Corning single mode cable to an EtherRegen. 

 

We aren't in the Objectivist forum. I related my ongoing experience of listening to ethernet cables upstream of fiber optic conversion, as was requested. People should test for themselves. Dismissing audiophile views with technical explanations doesn't have a very credible history IME, after decades as an audiophile. 

[barrows]

1 hour ago, jabbr said:

That was specifically considered in the 1990s and ensured against in the millenial era 10 Gbe standard ancient history. Since you know that this might occur, why wouldn't he have designed the equipment not to propagate jitter and other types of noise?

I do not understand the question: You vs He?  I am not John Swenson.  

As I understand John's theory, is has to do with the audio DAC, and not with the Ethernet receiver specifically.  If John's theory is correct,  (Ethernet)clock phase noise is affecting DAC conversion and creating audible problems.  Additionally, this has nothing to do with 10 Gbe, as most person's systems we are talking about are 1 GBe.  It is fine with me to question John's theory, it is not mine, and I presented it as such by saying: "if one ascribes to..."  I myself am skeptical, although I cannot deny that here, there was an obvious, subjective improvement in SQ when adding the Sablon cable from the switch to the oMd.  As the designer of this cable states that what it does is reduce noise, the only (speculative) reason for the improvement is that less noise into the oMd is allowing the Ethernet clock in that device to run with less noise.  The only thing I "know" is that even in the context of an optical ethernet system, and upstream Ethernet cable change was audible: perhaps this change would not be audible given a 10 Gbe set up... 

[Superdad]

1 hour ago, jabbr said:

That suggests a serious fault in your FMC/switch, either its 1Gbe and not tested, or if its 10Gbe+ then its not compliant with testing. 10Gbe+ compliance testing does not allow upstream noise/jitter to be propagated by a complaint device.

 

1 hour ago, jabbr said:

This article from 2003 gives a fairly English language description and pointers to the actual standards:

https://www.lightwaveonline.com/optical-tech/article/16649441/10gigabit-ethernet-devices-stressed-by-new-test

 

To quote from the article you so often promote:

"The receiver under test must be able to recover the signal error-free when a certain amount of sinusoidal or phase jitter is added to it, as specified by the standard. "You run the jitter all the way out to about 40 MHz," says LeCheminant, "because the typical loop bandwidth of the receiver is about 4 MHz. It has to be able to tolerate a reasonable amount of jitter above and beyond this 4-MHz rate."

 

All in service of assuring low BER (bit-error-rate) which is fine.

However, it is phase-noise at very low offset that is causing ground-plane bounce--and unlike very high frequency jitter, which seems to get canceled right under the traces, the low frequency stuff propagates on the board rather easily.

 

1 hour ago, jabbr said:

That was specifically considered in the 1990s and ensured against in the millenial era 10 Gbe standard ancient history. Since you know that this might occur, why wouldn't he have designed the equipment not to propagate jitter and other types of noise?

 

The "he" you seem to be referring to is John Swenson. If so, then indeed the entire architecture of the EtherREGEN (12-core-per-port magnetics with center-taps tied to ground; ultra-low-noise power networks, active-differential digital isolators with ultra-low-jitter flip-flops on both sides of the "moat," low-phase-noise clock built in plus input for external reference clock) is designed to not propagate the electrical (not data) perturbations we have determined are infiltrating Ethernet and making their way to the DAC. 

[Superdad]

9 minutes ago, barrows said:

It is fine with me to question John's theory, it is not mine, and I presented it as such by saying: "if one ascribes to..."  I myself am skeptical, although I cannot deny that here, there was an obvious, subjective improvement in SQ when adding the Sablon cable from the switch to the oMd.  As the designer of this cable states that what it does is reduce noise...

 

Hi Barrows:

I get a kick (sort of) out of how are always so vigorously skeptical of what John (who literally defined the power networks inside hundreds of very large and small scale ICs-- including Ethernet PHYs--for decades) has been explaining in various ways for years; Yet you'll just accept the word of some subjective boutique gent that a $400 Ethernet cable is "reducing noise"--because you heard a difference.  It's bit like my friend who believes his homeopath over his doctor--even though he takes the meds his doc prescribes. 

 

[And for those not already aware, the oMd Barrows refers to is the Sonore opticalModule Deluxe fiber media converter--also designed by John Swenson, and sounding better than ever in its latest iteration, thanks to carefully chosen, scientifically-based circuit tweaks. It's likely the world's best "sounding" FMC. But it does not have the EtherREGEN's costly "moat' of active-differential digital isolators and ultra-low-jitter reclocking flip-flops--which will be Gigabit in the forthcoming Gen2 model.]

[jabbr]

30 minutes ago, Superdad said:

 

 

To quote from the article you so often promote:

"The receiver under test must be able to recover the signal error-free when a certain amount of sinusoidal or phase jitter is added to it, as specified by the standard. "You run the jitter all the way out to about 40 MHz," says LeCheminant, "because the typical loop bandwidth of the receiver is about 4 MHz. It has to be able to tolerate a reasonable amount of jitter above and beyond this 4-MHz rate."

 

 

I'm promoting the 10GBASE-X standard but its technical. The article is merely a convenient and brief English language explanation.

 

30 minutes ago, Superdad said:

All in service of assuring low BER (bit-error-rate) which is fine.

However, it is phase-noise at very low offset that is causing ground-plane bounce--and unlike very high frequency jitter, which seems to get canceled right under the traces, the low frequency stuff propagates on the board rather easily.

 

The fact is that these things are measurable and are actually measured by companies that do high speed digital design. You've never published measurements nor even against the "stressed receiver" standard tests.

 

The simple fact is that at 100Gbe, *any* jitter causes a train crash. An entire 1000 bit Ethernet frame is basically the same length in time that a single 100M bit is long and the timing requirements are very tight. If noise propagated down the system then the internet  simply would crash and burn.

 

So no "low frequency stuff" doesn't easily propagate down a modern network board rather easily.

 

30 minutes ago, Superdad said:

 

 

The "he" you seem to be referring to is John Swenson. If so, then indeed the entire architecture of the EtherREGEN (12-core-per-port magnetics with center-taps tied to ground; ultra-low-noise power networks, active-differential digital isolators with ultra-low-jitter flip-flops on both sides of the "moat," low-phase-noise clock built in plus input for external reference clock) is designed to not propagate the electrical (not data) perturbations we have determined are infiltrating Ethernet and making their way to the DAC. 

 

I'm talking about whether upstream copper Ethernet cables matter. I'm assuming John Swenson designed the device @barrows was referencing. I'm assuming the device is 1Gbe capable. I am taking his statement that the upstream cable matters. I am saying that the issue was considered and dealt with now 25 years ago.

 

FWIW I have personally tested a wide variety of 10Gbe,40Gbe and 100Gbe NICs, cables and switches upstream of my endpoints. I can't hear a difference. This thread documents pretty much every device, SFP, switch, cable etc etc I have or am using. ...hmm now that we are talking about 100GBASE ... also these QSFP28 modules https://convergedigest.com/source-photonics-debuts-100g-lr4/

[jabbr]

32 minutes ago, Superdad said:

However, it is phase-noise at very low offset that is causing ground-plane bounce--and unlike very high frequency jitter, which seems to get canceled right under the traces, the low frequency stuff propagates on the board rather easily.

 

 

What does this even mean for  800Gbe? You are talking about phase noise for audio signals and DACs in the same way you are talking about phase noise for Ethernet clocks ... really 10hz offsets don't even make sense. This could be something that @Miska etc could be concerned with if he cared about the rate at which packets are issued ... if this level of packet bursting causes ground plane bounce in endpoints attached to the DAC ... and indeed it might ... then the problem needs to be addressed at the TCP vs RTTP etc level. Maybe RAVENNA matters. Its not Ethernet nor Infiniband problem with modern networks (e.g designed after Y2K).

[jabbr]

48 minutes ago, barrows said:

I do not understand the question: You vs He?  I am not John Swenson.  

As I understand John's theory, is has to do with the audio DAC, and not with the Ethernet receiver specifically.  If John's theory is correct,  (Ethernet)clock phase noise is affecting DAC conversion and creating audible problems.  Additionally, this has nothing to do with 10 Gbe, as most person's systems we are talking about are 1 GBe.  It is fine with me to question John's theory, it is not mine, and I presented it as such by saying: "if one ascribes to..."  I myself am skeptical, although I cannot deny that here, there was an obvious, subjective improvement in SQ when adding the Sablon cable from the switch to the oMd.  As the designer of this cable states that what it does is reduce noise, the only (speculative) reason for the improvement is that less noise into the oMd is allowing the Ethernet clock in that device to run with less noise.  The only thing I "know" is that even in the context of an optical ethernet system, and upstream Ethernet cable change was audible: perhaps this change would not be audible given a 10 Gbe set up... 

 

I can't make this point enough: the reason that I've been discussing 10Gbe/40Gbe/100Gbe is not that they are needed for audio, rather that the respective standards and noise requirements are such that propagated noise is not allowed.**

 

** replace not with a sufficiently low quantity however 100Gbe is approaching physical limits of low, and the network world is up to 800Gbe. 100Gbe is now sufficiently cost effective that other folks are able to replicate the systems that I've described here in detail.

[SQFIRST]

Finding the discussion really interesting and I think I am more impressed with @jabbrs debating skills, as good as their technical grasp is. I have gotten some very good pointers from jabbr that set me in the right direction and I only speak to things that I actually tried myself. I do see a benefit of 10g isolation now as well as balanced power isolation. However, I need to say that myself, and I assume a lot of audio lovers that are getting into serious streaming, do not have the kind of robust network (10G and up) or aspirations of one. Everyone will get to their optimum in a different manner. Products like the EtherRegen helps the average user as it boosts their audio/network quality, general understanding and if nothing else, sets them on the right path. Information from this thread is valuable but it can get confusing without the correct orientation and understanding. It comes down to curating the appropriate information and the fun of trying things.

Thanks all for a great discussion!

[jabbr]

2 hours ago, audiobomber said:

Everything here is 1Gbe. The switch is a Silent Angel N8, with linear power supply. The FMC is a TP-Link MC220L, SFP's are Finisar 1318 with Corning single mode cable to an EtherRegen. 

 

We aren't in the Objectivist forum. I related my ongoing experience of listening to ethernet cables upstream of fiber optic conversion, as was requested. People should test for themselves. Dismissing audiophile views with technical explanations doesn't have a very credible history IME, after decades as an audiophile. 

 

To be clear, although this thread has been opened to discussion of SQ comparisons in optical network equipment, the original motive for starting the thread was to document working configurations and best practices of fiberoptic ethernet equipment. Early on (circa 2015) we discussed that the TP-Link MC220L may not be perfectly isolating, and I haven't used them since that time. That said supplying with linear power is a very good idea particularly when in the audio area. Fiber isn't magic.

 

I've never dismissed all audiophile views, that said, we are comparing apples to oranges. I have explicitly stated the circumstances under which upstream copper cables don't affect SQ. and envision situations where they might affect SQ. I've done my own listening tests. The fact is that 100Gbe equipment is now <$1000 and 10Gbe equipment is now <$200. At the end of the day we are talking about Ethernet equipment which is by definition technical. 

 

I'm not dismissing what you hear. I am dismissing some technical arguments, or providing links to clearcut ways to test these arguments.

[barrows]

2 hours ago, Superdad said:

Hi Barrows:

I get a kick (sort of) out of how are always so vigorously skeptical of what John (who literally defined the power networks inside hundreds of very large and small scale ICs-- including Ethernet PHYs--for decades) has been explaining in various ways for years; Yet you'll just accept the word of some subjective boutique gent that a $400 Ethernet cable is "reducing noise"--because you heard a difference.  It's bit like my friend who believes his homeopath over his doctor--even though he takes the meds his doc prescribes. 

I am skeptical because the way JS has explained it, the following question arises, which I have asked before, but no one has actually attempted to answer:  If accumulated Ethernet clock phase noise of the entire Network chain is causing audible problems in the way in which John describes, then it appears that it would follow that audio streamed from Tidal, Qobuz, etc, should be seriously de-graded, considering the number of Ethernet devices, all with clocks contributing phase noise, a streamed track passes through on the way from the Qobuz server to one's DAC.  Additionally, this proposed noise, must make it's way from the Renderer, to the ground plane of the DAC's clock to have an audible effect by this theory, and considering the case of galvanically isolated USB input, through the isolation of the USB receiver.  Now I understand that USB isolation as such is not absolute (GMRs, Opto-isolators, etc), but it is quite good.  I have never said John is wrong, just that I am skeptical that his proposed theory (never objectively supported either, at least publicly) actually describes what may be going on.  I accept the possibility.

As for the Sablon Ethernet cable, I do not trust the designers description without my own investigation: the cable audibly improves performance in an obvious way-not just different sounding, but clearly an improvement.  My understanding is that this cable is built from an ordinary, high quality, industrial Ethernet cable base (likely Belden or something like that), which is then improved by the addition of noise reduction compounds: it does not rely on wires with "magical" properties, or some out of specification weird non-standard Ethernet cable geometry, or any other audiophile BS which some cable makers claim.  I am quite familiar with the effects of various noise reduction schemes and compounds aimed at reducing high frequency noise (damping), such as ferrites, various crystalline elements, etc. from my own use of these materials.  As this cable makes an obvious improvement, which I have verified myself, to me the obvious conclusion to deduce is that the improvement is due to noise reduction.  Reduced noise transmission, from the switch to the oMd is the only reasonable conclusion as to why this cable makes a difference.

[taipan254]

1 hour ago, jabbr said:

 

To be clear, although this thread has been opened to discussion of SQ comparisons in optical network equipment, the original motive for starting the thread was to document working configurations and best practices of fiberoptic ethernet equipment. Early on (circa 2015) we discussed that the TP-Link MC220L may not be perfectly isolating, and I haven't used them since that time. That said supplying with linear power is a very good idea particularly when in the audio area. Fiber isn't magic.

 

I've never dismissed all audiophile views, that said, we are comparing apples to oranges. I have explicitly stated the circumstances under which upstream copper cables don't affect SQ. and envision situations where they might affect SQ. I've done my own listening tests. The fact is that 100Gbe equipment is now <$1000 and 10Gbe equipment is now <$200. At the end of the day we are talking about Ethernet equipment which is by definition technical. 

 

I'm not dismissing what you hear. I am dismissing some technical arguments, or providing links to clearcut ways to test these arguments.

 

This is a really interesting discussion. I have a few questions that I hope aren't OT:

 

1. @jabbr can you please talk more about your endpoint? It looks like it is running off a 2.5G optical connection? How do you power it? What's the OS running the NAA? Obviously 2.5G is below your 10Gbe threshold, so by the standards you're discussing the signal from switch to endpoint may not be immune to jitter. Is this conclusion incorrect?
2. @jabbr Do your conclusions extend to running 10GBbe over CAT6A instead of Fiber? In other words, is CAT6A also immune to jitter? Is it impacted by EMI / RFI? Or is the cable shielding (assuming the cables are made to spec) sufficient to prevent any timing / quality issues? 
3. Can errors, jitter, or noise from the server / dirty side be embedded in the signal such that they get encoded in the optical stream to the endpoint, and then to the DAC? Or is that impossible? There's obviously a big "server first" crowd here and at other forms (the Taiko fans, the DIY server fans) and am curious how the thinking here about "clean" and "dirty" sides reconcile with the "server first" view that "everything in the chain matters".