Optical Network Configurations

[jabbr]

I have successfully used a gigabit ethernet connection between a Netgear Switch (1000base-T) to a TP-Link MC220L fiber media converter (FMC) over LC-LC Duplex OM3 fiber to MC220L to DN2800MT motherboard (NAA).

 

SFPs successfully used with MC220L:

Finisar FTLX8571D3BCL (supposed to be 10g but works ...)

GLX-SX-MM-TO (Cisco compatible labeled "iNet")

[jabbr]

iPower on order

[jabbr]

I use force when FMC to FMC and otherwise auto.

 

I ordered from musicdirect.com and am still waiting ... still says 4-6 weeks

[jabbr]

So far, I installed the Intel x520-da2 NIC into both a Ubuntu based NAS using the Intel ixgbe driver which I installed using the allow_unsupported_sfp=1 flag, as well as my OS X 10.10.3 desktop machine. I used the Small Tree driver for OS X. In order to get this working I used a FakePCIID.kext extension (you need FakePCIID.kext and FakePCIID_Intel_GbX.kext).

 

I'm going to start laying fiber -- these are 100m LC-LC duplex OM3 rated cabled I got off of ebay -- turned out to be Corning

 

Running 4 cables from 2nd floor office to basement.

[jabbr]

Got sidetracked looking at porting the Intel IXGBE driver to OS X ... umm maybe when I get some extra time over the summer :\

[jabbr]

Inserted FMC's* into a simple home set up of control and audio pc (JPLAY/ AO ) with static IPs and direct ethernet connection. Local HDD and no NAS. Control PC is laptop with one ethernet port (to audio PC) and connects to router wirelessly for internet duties.

 

Wasn't able to connect with the FMC's in between PC's, so removed FMC's and tried to reconnect (desperate to hear music by then) but still wouldn't work. Tried the usual JPLAY troubleshoots, even different static and dynamic IP- not sure what I was doing wrong.

 

In desperation, changed to single pc mode with FMC's positioned in between cable router and one PC- hey presto- MUSIC.

 

First thing that hits you is deeper bass and a greater sense of ease. Music becomes more analogue and natural sounding. Amazing tweak for the money (more than a tweak actually) and highly recommended!

 

Not sure why FMC's wouldn't work in dual PC mode. Maybe I hadn't set up my static IPs correctly, even though the ones selected worked fine before inserting FMC's. Would appreciate any help with static IP set up.

 

 

 

 

 

*2 x MC 220L's, 2 x GLC-SX-MM 30-1301 (used), 1 x 50/125 LC-LC Duplex OM3. Using 1 x Teradak U9VA (set to 5V) to power one FMC.

 

Thanks to Jabbr for help with purchasing decisions.

 

The FMCs take 9V, correct? So perhaps your Teradak set to 5v is causing a problem?

[jabbr]

Just connected my 220L to an Anker Astro E5, and works fine. Not sure how many amps each Anker port outputs

 

The U9VA is variable up to 9V 1A from its 2.1mm DC output, but I used its (additional) 5V USB output instead. As Ted illustrated, the 220L manual says that 5V is okay, but doesn't mention amps.

 

Jabbr, the issues I had with dual pc cannot be anything to do with how the 220L's were powered because they worked when moved in front of single pc (with identical powering).

 

Right. I don't have that setup so its really hard to know. Your equipment is working (because you tested that). The network setup is a bit complicated where you have two networks that your control PC sits in between and connects to both. Since it wouldn't work even after you reconnected with copper directly, its more likely to be a network issue than something to do with the FMCs. One of the advantages of a fiberoptic switch is that it simplifies the network topology while retaining optical isolation.

[jabbr]

Success: The TP Link SG5412F is a great switch for the price. It is fanless & silent. It has 12 SFP 1 gigabit ports and 4 shared with RJ-45 so it is very flexible.

 

I've successfully connected using an Intel x520-DA2 card with an Intel SFP+. The TP Link is not picky about brands of SFP modules. Using some from Foundry Networks. I would say that the only disadvantage I see over the Diablo 8 port SFP switch it that the Diablo accepts a 9V input so much easier to LPS whereas the TP Link has an AC input.

[jabbr]

Is your NAS a Synology? I saw that your NIC is supported on some of their NAS devices.

 

My NAS is a low powered Xeon (E3-1265L) running Ubuntu/ZFS with an LSI 9300 SAS3 card and an Intel x520-DA2 card. The external SAS3 enclosures are from PC-Pitstop. I have an older NAS running openindiana/ZFS in a supermicro enclosure.

[jabbr]

NAS:

Xeon E3-1265L (or equivalent)

H100 watercool

Motherboard: Supermicro | Products | Motherboards | Xeon® Boards | X10SLH-F -- has IMPI ... important!

SAS3 Enclosure: "+getMessage("iPrintVerKit")+"

SAS3 HBA: http://www.lsi.com/products/host-bus-adapters/pages/lsi-sas-9300-8e.aspx

NIC: ARK | Intel® Ethernet Converged Network Adapter X520-DA2

[jabbr]

that's IPMI (not sic IMPI) which is very useful for managing servers in another area e.g. basement

[jabbr]

VDX-6720-24 is a 24 port 10g/1g switch. When I turned it on, it was terribly loud, but thankfully after it boots, it gets quiet (enough that I'm not going to swap out the fans). It is the real deal big-iron of yesterday switch -- these are available now that the big datacenters are moving to 40g and 100g backbones. Get this only if you have technical expertise (I found a great price on ebay). I got this for video editing and playback.

 

It is working with Brocade compatible 10g-sx and Brocade/Foundry 1000base-sx SFP modules. I connect it to my copper network through a Netgear switch that has a 10g uplink.

[jabbr]

Netgear GSM7328T is a 24 port stackable ethernet switch which has 4 SFP slots (1gbe) and accepts up to 4 modules. I have a 10G SFP+ module which is used as an uplink to the 10G switch (Brocade VDX-6720 above)

 

Dell c/w Netgear AXM761 10G SR SFP+ module connects to Brocade VDX-6720 with Dell c/w Brocade 10G-SFPP-SR SFP+ module

 

I would get this only if you have a good sized copper network and need an inexpensive 10g uplink and the price is right.

[jabbr]

Well, my Ankers (E4 and E5) are first gen and therefore only 2A (and 1A, but tried 2A port). Thought nothing of it; so my USB to 2.1MM dc plugs arrived so I swapped out my MC220L stock wallwart on my switch-side FMC and inserted the E5. ...

 

E1 and E5 2nd Gen both work ... unfortunately 2nd gen doesn't power while charged -- sigh :-\

[jabbr]

I will let jabbr weigh-in on that. I do not want to say, because I do not understand this subject completely, but another thing I read on Diablo web page is that it can take SFP modules of SX, LX and LH fiber types.....I thought we were using (in multi mode at least) LC-LC type of fiber.

 

Just want to make sure, because this switch is about $300. Thanks.

 

Yes the Diablo should be able to take 1000base-T SFPs and thus connect to copper Ethernet. I have the TP-link SG5412F which is a great switch (described in previous post). ~$250 new Advantage: 4 shared SFP/RJ-45 ports so can do 4 copper ethernet without SFP modules, has 12 total ports. TP does AC power which is a disadvantage if you want to LPS (not sure how important that is).

 

In any case you can get either to work.

[jabbr]

There's also the TP-Link TL-SG3424 which has 24 copper and 4 SFP ports for ~$160.

 

For my own setup, I have entirely optical switches which are connected to the copper network via an optical uplink, but this is all very very configurable and there are many options to suit each person's situation. The point is that common perception has been that optical networking is too expensive for home use. I think the prices are, in fact, very reasonable and certainly far less than people spend on fancy interconnects. There should be no reason for a super fancy copper Ethernet cable to exist when Corning fiber is so easily available.

[jabbr]

?? OK. I'm officially flumoxed. Yes, as I stated numerous times, the plugged in Ankers work fine for SSD power. But Jabbr confirmed that recharging them while using them for these FMCs does not provide enough power......but then you counter that. I tested my E4 and E5 (first gen) both umbilicaled and not, and in both cases do not power "enough" to light all four link/power lights. Any theories as to what's going on here?

 

No its not that they don't provide "enough" power while being charged: The official Ankers FAQ states that Gen2 devices do not provide (any) power while being charged. The Pro2 device does and the 1st gen devices do. I think its an electronics issue.

[jabbr]

A big thank you for this solution Jabbr!

 

Well I hope it is a solution These ideas are all based on ideas from the forum. I certainly didn't invent optical networking. Perhaps just noticed that the prices have become reasonable for home users with specific requirements ... and then there is ebay

 

Another experiment:

 

I really think that fiber for file transfer, to a low power, low EMI memory cache that, bypassing SPDIF and USB, serves the translated audio file of any format gaplessly via I2S (asynch and DAC master-clocked) into the DAC is the way forward to lowering digititus to perhaps an indistinguishable level. Forget Mac or Windows as they are too bloated, so it has to be Linux or firmware. Still need to use an iPad wirelessly with OHMedia type of controller for ease of use. Pipe dream?

 

Not at all. There is no reason why we couldn't use the LVDS version of I2S/raw DSD. Would need support from software like HQPlayer/NAA as well as hardware (starting to happen). Presumably the I2S output of the Amanero board which feeds the DSC1 would be a point at which the raw DSD signal could also feed the DSC1 ... this is open hardware so modifiable, but other DACs could follow suit.

 

My digital has never sounded better, and there are still refinements possible, I know.

 

The setup is: 2x TP Link MC210CS with a Lynn SC (9/125) fiber cable linking the NAS to the switch/wireless bridge (powered by an LPSU) on which the streamer is connected via LAN cable. Using the IFI low noise wall-wart and a Teddy Pardo all additionally filtered to power the FMC. All cables are BlueJeansCable certified.

 

Moving from wireless was improved by wired NAS to streamer via switch. And this fiber link improved it further.

 

Open items:

 

1. I was not able to get the streamer to connect to the switch via the FMC, so that needs to be resolved

 

There might be some complexity in your network. Are you using your wireless router to bridge two wired networks, or does it provide access for wireless clients, or are you doing both? Does your network have more than one path between devices? It might be getting "confused".

 

When moving to optical switch: you would need a SC-SC to LC-LC patch cable (singlemode 9/125) and an 1000base-LX SFP module.

2. Need to replace the copper switch with a fiber switch also on linear power.

3. Need to replace the IFI wallwart with linear for peace of mind.

4. Currently using a Synology NAS, but Jabbr's custom one with a fiber NIC would logically be better. Synology does have a few expensive ones with a fiber NIC or a slot to add one, so that is another option.

5. Find a way to directly link streamer to NAS via fiber, without switch in between, and still have the NAS on the wireless network.

 

Cheers

 

This suggests that you have a single wired network, and place wireless router to provide access for wireless devices to entire network. So connect NAS to switch, streamer to switch and wireless router to switch. Now replace ethernet between streamer to switch with FMC pair and intervening optical and this should work?

[jabbr]

Thanks, that is one of the things I am curious to try as that is close to how the setup is. The gigabit switch (to which NAS and streamer are connected is also a wireless bridge (WD My Net AC bridge), which is connected wirelessly to my main home's wireless router. This way internet radio and iPad UPnP/OHMedia controllers are possible. Although I think even better would be a NAS with fiber NIC via SC cable to FMC to super short LAN cable into streamer. I would need to figure out how to get the NAS on wireless network also. The latter is something I also want to try on my current Synology since it has 2 ethernet ports.

 

Cheers

 

You have a wireless router (bridge). Routers are a bit different that switches and although the traditional distinctions have become very blurred recently, traditionally a 1Gbs switch will maintain a 1Gbs connection between any two ports even when many pairs of ports are being concurrently used whereas a router shares the total traffic. The router however can direct traffic between two distinct networks -- in your case between the wireless and wired network -- or typically between a home network and the internet (e.g. cable modem router).

 

With a switch there is no need to worry about direct point to point Ethernet connections. Your NAS has two Ethernet ports both of which should be connected to a switch and which would allow two different clients each to have an independent 1gbs connection to the NAS. Using link aggregation, the NAS would present a single IP address, and the switch/NAS would connect two different clients to each 1gbs connection automagically. The advantage of a fiber switch is that you need 1/2 the number of FMCs because each fiber cable connects to the switch on one end and an FMC on the other (or directly to a fiber enabled client). So, connect the NAS to the switch and the switch to the wireless router and the NAS is "on" the wireless network.

 

Now what might be wrong? Hard to say. I would try using the same subnet for both your wireless and wired networks (i.e. all addresses in range 192.168.1.0 - 192.168.1.255) Do you also have another router to connect to internet? Which device is assigning IP addresses (DHCP server)?

[jabbr]

jabbr: After I installed Mallanox fiber NIC in my PC, it seems that PC is trying to boot on the NIC. I get a message saying...Mellanox...iPEX....Boot...., etc. and after few seconds it goes to normal boot sequence. Is that something normal? I am waiting on my fiber switch before I connect this NIC to switch. Thanks.

 

That's a good thing if you are trying to boot disk less over the network

 

Just go into your BIOS and switch the boot order to look for a hard drive first

[jabbr]

This thread (and similar) tells me that the Ethernet packets behave much the same way when it comes to SI and PHY issues as per USB. If the dynamics are changed in the transmission media or the optical to copper converters then what you hear will change.

 

What's needed is an EthernetRegen or RegenE

 

It is very similar. Each fiberoptic connection takes an electrical signal and converts to optical and then reconverts to electrical. The copper ethernet may need to push a signal down 100m or so of copper and consequently needs power. The signal itself gets degraded and needs to be reconstituted. Fiberoptic has some of the same issues but much less, and does not need the same increase in power as the transmission length increases. A model where long distance connections are fiberoptic switch to switch and using LR singlemode optics, and short connections are computer to switch using SR multimode would then involve 3 serdes opticoelectric "regens". The point is that the intervening fiberoptic switch actually improves the signal and reduces the work of the computer's NIC. The Xeon can move the regenerated digital information directly into processor cache, entirely avoiding the copper ethernet PHYs.

 

That's the theory. The proof is in the sound and at least some initial impressions are supportive.

[jabbr]

I checked BIOS on my computer with Mellanox fiber NIC, two things have happened since I installed this card.

 

Under boot devices, now I have Mellanox card as device, which wasn't there.

 

Whenever I reboot this computer, it first wants to boot on Mellanox fiber NIC, and after few minutes goes to regular boot sequence, even though it is not checked under boot list, only HHD and USB are checked.

 

It does not bother me, but if it has any significance, please let us know. Thanks.

 

The card has firmware to boot off the network (PXE). You can reorder the boot devices so that the BIOS looks for the hard drive first. No harm done.

[jabbr]

Another Mellanox issue for me: I can't get it to see a cable. It shows as an ethernet adapter in my network on my Zuma (along with my copper one which I need to keep connected so I can see my computer on remote desktop....at least until the fiber/Mellanox works). But it has a red X and says network cable unplugged. The other end (TP-Link MC220L) has the FX link blinking, not steady (as my successful NAA FMC-to-FMC fiber install is...steady). I've tried different fiber cables, different SFPs in the card, sdifferent FMCs. everything. TCPIP settings seem ok (DHCP, etc). I have driver 4.1.10092 (found automatically). I tried setting the FMC to auto, to force, resetting in between, nothing.

 

Any ideas? Maybe the HQP pc server end isn't worth it? Stay copper for now as long as NAA is fiber??

 

Update: I upgraded the driver to 4.8 (supposedly needed for WS2012) but still same status...seen but showing network cable unplugged. Some googling says I need a subnet? Argh...over my head here.

 

I'm not a Mellanox expert but perhaps your SFP module is not compatible with the card?

[jabbr]

I installed (physically) Mellanox in the slot again and rebooted twice. My card is working I have connectivity, but no remote desktop. Argh....

 

There are too many settings under Advanced tab of Mellanox Properties, so hopefully it might be one of them.

 

I will google and see. I know jabrr has some family engagements today, so will wait for him to get back.

 

I gave an admin password to BIOS configuration; hoping that Mellanox will not be able to add itself to BIOS boot sequence, but still it did, I unchecked, but it still wants to boot on that first, then goes to HDD.

 

Cannot connect all three; I am short one SFP. I thought Mellanox would not need it but it does.

 

If it's just Remote Desktop not working then it must be some network config issue eg firewall, maybe Remote Desktop is looking for a different interface etc etc

[jabbr]

Remote Desktop works when I give an IP address to connect, instead of giving it a computer name, as I used to connect. I will have to give this card a static IP. Thank you guys.

 

You can probably configure for DHCP