Optical Network Configurations

[The Computer Audiophile]

Or, can you recommend a good 10Gb card that will work with these modules?

[The Computer Audiophile]

This one?

Intel Ethernet Converged X710-DA2 Network Adapter (X710DA2)

https://www.amazon.com/Intel-Ethernet-Converged-X710-DA2-X710DA2/dp/B00NJ3ZC26/

[jabbr]

48 minutes ago, The Computer Audiophile said:

Or, can you recommend a good 10Gb card that will work with these modules?

 

52 minutes ago, The Computer Audiophile said:

Hi @jabbr, will this Intel card work with the Ubiquiti UF-SM-10G-S BiDi SFP+ module?

 

Intel 82599EN Single-Port 10G SFP+ PCIe 2.0 x8, Ethernet Network Interface Card - https://www.fs.com/products/75851.html

 

Ubiquiti UF-SM-10G-S - https://www.ui.com/accessories/fiber-modules-cable/

 

Ok so first off, the first card is NOT an Intel NIC rather an OEM NIC that uses the Intel 82599EN ASIC/controller ... so it gets tricky, the Intel driver enforces a check that requires an Intel chipped SFP(+) ... for the most part but it gets tricky

as Intel says:

https://www.intel.com/content/www/us/en/support/articles/000007045/network-and-i-o/ethernet-products.html

(and that's how I started using the Finisar SFP(+) modules BTW ... but note that they are coded for Intel...) ... look for the code: E10GSFPSR or ...LR

 

Now usually you can use fs.com SFP modules and check "Intel compatible" when ordering so fs.com will code it for Intel

 

Solarflare and Mellanox are less picky about modules, I probably wouldn't try using a Ubiquiti module in an Intel NIC because ... who knows ... those Solarflare NICs which ... are available  are really sweet if you can get one for a reasonable price ... I have the "Flareon Ultra" 

 

https://www.amazon.com/Solarflare-Communications-10Gigabit-Ethernet-SFN7322F/dp/B00G2HUUDW/ref=lp_7643221011_1_7?srs=7643221011&ie=UTF8&qid=1592340724&sr=8-7 ...

 

or "open box" on eBay: https://www.ebay.com/itm/New-SolarFlare-SFN7322F-Dual-Port-10GbE-PCIe-3-0-PTP-Server-I-O-Adapter/372905749905?hash=item56d2ea2d91:g:zpEAAOSwJ4peFQGh

 

or if you prefer to pay 10x-20x more for list price 😳 : https://www.insight.com/en_US/shop/product/SFN7322F/SOLAR FLARE/SFN7322F/SolarflareFlareonUltraSFN7322F-Networkadapter-PCIe3.0x8-10GigabitSFP+x2/

 

these have special low phase error clocks which exceed the 10Gbe specs and are ultra ultra low latency (for 10Gbe) designed for the high speed traders where latency = $$$$$

 

Ok back to the fs.com "Intel" NIC, it will probably work fine for most purposes but it often WILL NOT run Intel firmware and that's why you need a genuine Intel NIC with the Intel YOTTAMARK on the back ... there are tons of offbrand NICs which use the Intel ASIC being sold as "Intel" ... very misleading IMHO

 

@plissken probably has practical experience with these issues?

[jabbr]

FWIW Solarflare and Mellanox both deal with NIC offload, low latency, DMA etc so Xilinx snapped up Solarflare and NVidia snapped up Mellanox as the datacenter war heats up.

 

Here is a PDF which has a list of SFP(+) modules tested by Solarflare ... not that the above Finisars that I've recommended and Avago (forgot to mention but also a stand up SFP OEM) are on their list ...

SF-103837-CD-19_Solarflare_Server_Adapter_User_s_Guide.pdf

[stray_cat]

On 6/15/2020 at 4:30 PM, jabbr said:

Great point. It’s the entire unit that has to be built to 10G in order to need to meet 10G standards. So a 10G switch can have a port run at 1G.

 

There can be power requirement issues, so while an SFP module should work in a SFP(+) port in general you cannot expect an SFP(+) module to work in an SFP port.

 

Short answer: no, longer answer: you could test and certify specific SFP modules for use in their equipment. Intel, for example, publishes a list of SFP(+) modules certified for the x520 NIC

 

The reason I started this thread was to allow people to post specific SFP/SFP(+) modules that work in specific equipment. 

 I've been working on absorbing all this. If I understand most of this advanced information correctly, including JS's discussion of residual "bad clocks" phase noise and OP's insights into the very high standards of 10gbe hardware and Finisar parts experiences, would ideal isolation be to maybe have a 10gbe switch with Finisar single mode 10gbe DFB laser transceivers (FTLX1471D3BCV) between a 10gbe router and the EtherRegen. (optical isolation between an Arris SB8200 cable modem & ER)? I'm thinking Mikrotik for both 10gbe HW boxes.

[jabbr]

9 hours ago, stray_cat said:

 I've been working on absorbing all this. If I understand most of this advanced information correctly, including JS's discussion of residual "bad clocks" phase noise and OP's insights into the very high standards of 10gbe hardware and Finisar parts experiences, would ideal isolation be to maybe have a 10gbe switch with Finisar single mode 10gbe DFB laser transceivers (FTLX1471D3BCV) between a 10gbe router and the EtherRegen. (optical isolation between an Arris SB8200 cable modem & ER)? I'm thinking Mikrotik for both 10gbe HW boxes.

To be clear: I have no direct information that 1Gbe fiberoptic Ethernet allows any significant noise transmission (including clock phase noise) however as of 20 years ago when these issues were being discussed and the IEEE standards finalized for 10Gbe, there were specific compliance testing (stressed eye pattern) to ensure that “bad clocks” and other noise does not pass via 10Gbe. That I know. 

[plissken]

21 hours ago, The Computer Audiophile said:

Or, can you recommend a good 10Gb card that will work with these modules?

 

Try https://www.ebay.com/itm/SolarFlare-SFN7002F-SFP-Dual-Port-10GbE-Flareon-PCIe-3-0-x8-Server-I-O-Adapter/303259728658?ssPageName=STRK%3AMEBIDX%3AIT&_trksid=p2057872.m2749.l2649

 

I've thrown Intel, Cisco, HPE tranceivers in them without any issue.

[The Computer Audiophile]

22 minutes ago, plissken said:

 

Try https://www.ebay.com/itm/SolarFlare-SFN7002F-SFP-Dual-Port-10GbE-Flareon-PCIe-3-0-x8-Server-I-O-Adapter/303259728658?ssPageName=STRK%3AMEBIDX%3AIT&_trksid=p2057872.m2749.l2649

 

I've thrown Intel, Cisco, HPE tranceivers in them without any issue.

Wonderful. Nobody on the Ubiquiti forum has responded to my question abut this yet. 

 

Edit: Ouch these are a little pricy. $19.95 +s&h :~)

[plissken]

2 hours ago, The Computer Audiophile said:

Wonderful. Nobody on the Ubiquiti forum has responded to my question abut this yet. 

 

Edit: Ouch these are a little pricy. $19.95 +s&h :~)

 

But they are audiophile and correction: The shipping is free 4 day CONUS.

[stray_cat]

On 6/17/2020 at 6:45 AM, jabbr said:

To be clear: I have no direct information that 1Gbe fiberoptic Ethernet allows any significant noise transmission (including clock phase noise) however as of 20 years ago when these issues were being discussed and the IEEE standards finalized for 10Gbe, there were specific compliance testing (stressed eye pattern) to ensure that “bad clocks” and other noise does not pass via 10Gbe. That I know. 

 

Cool, Thank you @jabbr.

 

I looked into it further and realized the 10gbe router might be too complex so ended up with 2 Mikrotik 10Gbe switches to run dumb and a couple of Finisar FTLX1471D3BCV modules to run SM OS1 fiber in between, kinda using them as high spec FMCs. The Tektonix stressed eye primer you posted the link to illustrates what you've been saying about the 10Gbe gear. I'm sure they're compatible with my chosen modules but will report back anyway.

[Markus87]

Hi @jabbr et al,

 

I am looking into upgrading my audio chain including the optimization of my home network for audio. I found this thread super helpful but still have some questions left.

 

Some background information:

• I exclusively stream music via Tidal hifi
• I have a 1Gbit/s synch. SFP based fiber connection (FTTH)
• I am in the build of a DIY HQP server w/ SFP+ network card
• I use a QNAP TVS-672n NAS @i7-8700T, 32GB ram w/ SFP+ network card // this is meant to run Roon core as HQP control in the future
• I am investigating AES67 DACs (RJ45 or better SFP based)
• I am targeting an optical network connection throughout the entire audio chain

My audio routing would be as follows: 

Tidal > HQP server (controlled by Roon on NAS) > DAC > amps > speakers

 

At the moment I have mainly two questions...

 

1) Concerning router: which option would be preferable?

 

A: setup pfsense on my NAS to use this as router/firewall (SFP+ i/o) in conjunction w/ a SFP+ switch (e.g. Miktorik CRS305) // possible noise advantage due to 10gbe spec compliance throughout the chain

 

B: usage of an optical router (Sfp i/o, e.g. Ubiquiti ER-12) in conjunction w/ a SFP+ switch // only 1gbe spec compliance but possibly easier to setup

 

2) Concerning the handling of other network devices: which option would be preferable? 

 

A: strictly separating them on different networks (I have cat8 in all rooms as well), shared devices as the NAS would use a separate eth per network, just one vlan routing to share internet connection and enable Roon control via wifi tablet

 

B: usage of vlans with all devices connected to the same switches as the audio gear

 

Thanks for any thoughts on an preferred setup! 

 

Cheers, 

Markus

[jabbr]

5 hours ago, Markus87 said:

Hi @jabbr et al,

 

I am looking into upgrading my audio chain including the optimization of my home network for audio. I found this thread super helpful but still have some questions left.

 

Some background information:

• I exclusively stream music via Tidal hifi
• I have a 1Gbit/s synch. SFP based fiber connection (FTTH)
• I am in the build of a DIY HQP server w/ SFP+ network card
• I use a QNAP TVS-672n NAS @i7-8700T, 32GB ram w/ SFP+ network card // this is meant to run Roon core as HQP control in the future
• I am investigating AES67 DACs (RJ45 or better SFP based)
• I am targeting an optical network connection throughout the entire audio chain

 

Nice. 
 

The “problem” with AES67 DACs is that they might not run NAA, they might. 
I would prefer a DAC that runs NAA or an endpoint that runs NAA and then outputs to USB DAC

5 hours ago, Markus87 said:

 

My audio routing would be as follows: 

Tidal > HQP server (controlled by Roon on NAS) > DAC > amps > speakers

 

At the moment I have mainly two questions...

 

1) Concerning router: which option would be preferable?
 

Do you mean switch? Router doesn’t really matter.

5 hours ago, Markus87 said:

 

A: setup pfsense on my NAS to use this as router/firewall (SFP+ i/o) in conjunction w/ a SFP+ switch (e.g. Miktorik CRS305) // possible noise advantage due to 10gbe spec compliance throughout the chain

 

B: usage of an optical router (Sfp i/o, e.g. Ubiquiti ER-12) in conjunction w/ a SFP+ switch // only 1gbe spec compliance but possibly easier to setup

 

2) Concerning the handling of other network devices: which option would be preferable? 

 

A: strictly separating them on different networks (I have cat8 in all rooms as well), shared devices as the NAS would use a separate eth per network, just one vlan routing to share internet connection and enable Roon control via wifi tablet

 

B: usage of vlans with all devices connected to the same switches as the audio gear

 

You can use VLANs if you want to limit broadcasts. Not sure how big a deal it is for 10Gbe network, but ok

 

You seem to be on the right track. Have fun!

[Markus87]

Thanks a lot @jabbr for your answer!

 

6 hours ago, jabbr said:

The “problem” with AES67 DACs is that they might not run NAA, they might. 

I would prefer a DAC that runs NAA or an endpoint that runs NAA and then outputs to USB DAC

Thanks for this hint! I will make sure it runs NAA/RAAT.

 

6 hours ago, jabbr said:

Do you mean switch? Router doesn’t really matter.

I meant router. The idea behind is that I stream Tidal and have a FTTH connection. So I am aiming for the longest possible fiber connection...

 

Maybe a stupid question: is it possible to connect the FTTH fiber transceiver to my 10GBe fiber switch (Mikrotik, no special coding required) instead of the router WAN? How would the connection between router and switch then look like (WAN/LAN) and what kind of routings would need to be made for this? My ISP requires an VLAN10 setting for the internet connection.

 

6 hours ago, jabbr said:

You can use VLANs if you want to limit broadcasts. Not sure how big a deal it is for 10Gbe network, but ok

To make it a little more tangible: Would the following two scenarios be equally good (from an audiophile POV)?

 

A:

NAS (Roon + photo storage) > SFP+ > OS2 > SFP+ > main 10Gbe fiber switch

Main 10GBe fiber switch > SFP+ > OS2 > SFP+ > secondary 10GBe fiber switch

Secondary 10GBe fiber switch > SFP+ > OS2 > SFP+ > HQP server (for Roon control)

Secondary 10GBe fiber switch > SFP+ to 10Gbase-T > cat7 > iMac (for photo storage)

no VLAN routings

 

B:

NAS (for Roon control) > SFP+1 > OS2 > SFP+ > main 10Gbe fiber switch

NAS (for photo storage) > SFP+2 > OS2 > SFP+ > main 10Gbe fiber switch

Main fiber 10GBe switch > SFP+1 > OS2 > SFP+ > HQP server (for Roon control)

Main fiber 10GBe switch > SFP+2 to 10Gbase-T > cat7 > iMac (for photo storage)

VLAN routings e2e

 

Main and secondary switch are in different rooms; iMac only has a 10GBase-T nic; rooms are connected via cat7 and OS2.

This is only a part of the chain to illustrate my question.

 

Does B make sense at all? Am I overthinking this?

 

Cheers,
Markus

[gadio4533]

On 6/3/2015 at 8:14 AM, John769 said:

Inserted FMC's* into a simple home set up of control and audio pc (JPLAY/ AO ) with static IPs and direct ethernet connection. Local HDD and no NAS. Control PC is laptop with one ethernet port (to audio PC) and connects to router wirelessly for internet duties.

 

Wasn't able to connect with the FMC's in between PC's, so removed FMC's and tried to reconnect (desperate to hear music by then) but still wouldn't work. Tried the usual JPLAY troubleshoots, even different static and dynamic IP- not sure what I was doing wrong.

 

In desperation, changed to single pc mode with FMC's positioned in between cable router and one PC- hey presto- MUSIC.

 

First thing that hits you is deeper bass and a greater sense of ease. Music becomes more analogue and natural sounding. Amazing tweak for the money (more than a tweak actually) and highly recommended!

 

Not sure why FMC's wouldn't work in dual PC mode. Maybe I hadn't set up my static IPs correctly, even though the ones selected worked fine before inserting FMC's. Would appreciate any help with static IP set up.

 

 

 

 

 

*2 x MC 220L's, 2 x GLC-SX-MM 30-1301 (used), 1 x 50/125 LC-LC Duplex OM3. Using 1 x Teradak U9VA (set to 5V) to power one FMC.

 

Thanks to Jabbr for help with purchasing decisions.

No familiar with those particular media converters but it could be referencing single or multi-mode fiber

[jabbr]

On 6/14/2020 at 7:14 PM, lmitche said:

The SQ difference between the fs.com and finisar SFP+s  is another story and is dependent on which DAC is in use. With one, there was little difference. With the other, it had a major impact on the sound adding audible distortions to the image. That led me to hypothesize that the EMI/RFI emissions impacted the SQ on the second DAC. These distortions disappear with the Finisar SPF+s.

 

I will stop posting here now.

No need to stop posting. I haven’t done as systematic listening comparison as you. I selected Finisar because major brand name vendors use them and everything is very well documented, so I know what I am getting eg laser type, specs etc.

 

I am curious, the SFP module effect is DAC dependent? Are you using an SFP enabled DAC?

[robi20064]

All,

 

I'm looking to simplify my current network setup which consists of a modem + router + 24 port HP switch ( the last dedicated for audio use). All the previous great talk here is driving me towards future proofing and going for SFP+ if possible. I was looking at the Ubiquiti ERX SFP, but that would be only SFP and gigabit, while the EdgeRouter Infinity would be a bit of an overkill for home use. 

 

Do you have any suggestion for an SFP+ capable router with 4-8 standard RJ45 gigabit ports and at least 2 SFP+ ports? A fanless design would be icing on the cake, however I'm not expecting a small footprint for such a device. The SFP+ would be connected to either an OM or an OpticalRendu for streaming.

 

Is the MikroTik CRS305-1G-4S+IN the only option here?

[ericuco]

1 hour ago, robi20064 said:

All,

 

I'm looking to simplify my current network setup which consists of a modem + router + 24 port HP switch ( the last dedicated for audio use). All the previous great talk here is driving me towards future proofing and going for SFP+ if possible. I was looking at the Ubiquiti ERX SFP, but that would be only SFP and gigabit, while the EdgeRouter Infinity would be a bit of an overkill for home use. 

 

Do you have any suggestion for an SFP+ capable router with 4-8 standard RJ45 gigabit ports and at least 2 SFP+ ports? A fanless design would be icing on the cake, however I'm not expecting a small footprint for such a device. The SFP+ would be connected to either an OM or an OpticalRendu for streaming.

 

Is the MikroTik CRS305-1G-4S+IN the only option here?

Although it is not fanless, QNAP QSW-804-4C might work. I used one for a while and fans are fairly quiet.

[robi20064]

Was looking at it before, I'm afraid it won't have routing capabilities, its not even a managed switch.

[jcn3]

2 hours ago, robi20064 said:

All,

 

I'm looking to simplify my current network setup which consists of a modem + router + 24 port HP switch ( the last dedicated for audio use). All the previous great talk here is driving me towards future proofing and going for SFP+ if possible. I was looking at the Ubiquiti ERX SFP, but that would be only SFP and gigabit, while the EdgeRouter Infinity would be a bit of an overkill for home use. 

 

Do you have any suggestion for an SFP+ capable router with 4-8 standard RJ45 gigabit ports and at least 2 SFP+ ports? A fanless design would be icing on the cake, however I'm not expecting a small footprint for such a device. The SFP+ would be connected to either an OM or an OpticalRendu for streaming.

 

Is the MikroTik CRS305-1G-4S+IN the only option here?

 

A Ubiquiti Dream Machine Pro would meet those requirements and is fanless -- router with 8 gigabit ports plus two sfp+ ports:  https://store.ui.com/collections/unifi-network-routing-switching/products/udm-pro

[R1200CL]

4 hours ago, robi20064 said:

Do you have any suggestion for an SFP+ capable router with 4-8 standard RJ45 gigabit ports and at least 2 SFP+ ports? A fanless design would be icing on the cake, however I'm not expecting a small footprint for such a device.

You can build your own fan less router that’s probably better than most and adding ipfire SW. 

https://www.iwill.no/en/iwill-ecolan-celeron-bay-trail-d-2-x-lan-2-x-sfp

This give you the options to use dedicated network for audio as well. 
You will still need a switch, as I think Linux firewall SW only support one interface out pr IP address. 
 

It’s very easy to setup. Download SW, and boot from USB stick, and install.

 

You can experiment with various mini pc and add required NIC’s as well as an alternative to readymade firewalls. 

 

[charlesphoto]

So I rolled SFP's - went from the MM TP-link's supplied with the opticalModules by Sonore, to Finisair FTLF1324P2BTV-B1 and single mode cable. The SFP's were purchased refurbished ($30 each) and the 4m cable from Amazon (Corning curve cable used). Using 10db attenuators on the send sides. Since I have to pull the cable through the wall and floor using the previous MM cable, there's really no way for me to A/B. My impression though is that the difference is subtle but there. Seemingly less 'glare' in the upper mids and treble, bass seems a bit better controlled, and overall a slightly less fatiguing listen. Worth it (the $70 total outlay)? Not 100% sure, though to appease my curiosity over MM vs SM SFP's - priceless! 

[charlesphoto]

Yeah, the longer I listen, these single mode Finisair's are better than the multimode I had before. Listening to one of my test albums, Monk's "Straight No Chaser" and it all just seems to hang together better. Cleaner. I was thinking my "new" mid-seventies Naim CB 160BD was probably in need of a recap, but it's just revealing more. Treble easier and cleaner now for this longtime tinnitus sufferer. Love these little tweaks, but oh lord, what kind of a crazy rabbit hole to go down! 

[plissken]

1 hour ago, charlesphoto said:

Using 10db attenuators on the send sides

You only need these for 40km optics and up.

[charlesphoto]

5 minutes ago, plissken said:

You only need these for 40km optics and up.

 

Ah, good to know. Will try without then. 

[jabbr]

On 7/10/2020 at 11:55 AM, charlesphoto said:

FTLF1324P2BTV-B1

 

These are compatible with 1000base-LX and as @plissken says don't need attenuators, for some reason some folks have used 1000base-EX modules (40km) and these do benefit from attenuation over short distances. FWIW I see no reason why EX modules might be better, indeed they use more power, so would avoid unless some additional info.