Optical Network Configurations

[89reksal]

13 hours ago, Bertel said:

15 hours ago, lasker98 said:

Mikrotik CRS305 and CSS610 are both 10Gb switches that have SFP+ ports. Both have settings for auto negotiation and manual link speed (10/100/1G/10G) selection. Unless I completely misunderstand how those settings should operate, I expect manually selecting 1G when using an SFP+ module in one of these switches to be able to connect to a device operating at 1G, whether the device is using SFP or SFP+ module.

 

Correct, that's exactly how my CRS305 works here

To clarify, you're stating that you have installed a non-dual rate SFP+ in one of the CRS305 SFP+ ports and have it communicate with a 1G device that has an SFP+ installed? If yes, you're the first confirmed case of someone successfully implementing this configuration. Please show screen shots of all relevant settings you've made in your Mikrotik OS, also showing the exact module used as shown in the OS. Hopefully this will allow myself and others to duplicate your success.

[89reksal]

3 hours ago, jabbr said:

I have a Mikrotik 10g switch and use it to connect to my 1g endpoints. I have been since the very day I installed it which was before I posted that it works...

I'm assuming you're again referring to implementing this using dual rate SFP+? Will you confirm that you believe your Mikrotik switch (CRS305 or CSS610) is working correctly even though it doesn't respond to the auto negotiation or manual link speed settings in the switch OS when using non-dual rate SFP+ in your switch when connecting to your 1G endpoints?

 

I'm trying to be as specific as I can with my questions in hopes of getting an equally specific reply.

[Bertel]

27 minutes ago, lasker98 said:

To clarify, you're stating that you have installed a non-dual rate SFP+ in one of the CRS305 SFP+ ports and have it communicate with a 1G device that has an SFP+ installed? If yes, you're the first confirmed case of someone successfully implementing this configuration. Please show screen shots of all relevant settings you've made in your Mikrotik OS, also showing the exact module used as shown in the OS. Hopefully this will allow myself and others to duplicate your success.

 

No, that's not the case (apologies if I didn't fully understand or communicate clearly). In my CRS305's four SFP+ ports, I have two connections at 10G and two at 1G. Of the two operating at 1G,

• one is the dual rate SFP+ module Finisar FTLX1475D3BCV,
• the other is a Finisar FTLX1475D3BCV, but that is a SFP module, not a SFP+

[Jud]

4 hours ago, Bertel said:

 

With the "ALL SFP+ modules will work at at 1Gb" statement, I was also wondering why e.g. the datasheet of the Finisar FTLX1475D3BTL SFP+ module specifically states "Supports 9.83 to 10.5Gb/s bit rates"

 

I think we probably have to be aware of context, depth of experience, and nuance here.

 

There are two different contexts we're talking about when we say "works at" a particular data rate:

 

(1) Whether a sending SFP+ module/switch can be set to only send SFP rates; or

 

(2) Whether a receiving SFP+ module/switch will accept a 1G data rate from a sending SFP module/switch.

 

I think it is in context #2 that John may have meant "work at" a particular data rate. This is happening in @lasker98's own setup. But since we have less experience than John, for whom the distinction is probably implicit, it was easy for us to take this (incorrectly) as being true of context #1 as well.

[89reksal]

2 hours ago, Jud said:

(1) Whether a sending SFP+ module/switch can be set to only send SFP rates;

 

2 hours ago, Jud said:

But since we have less experience than John, for whom the distinction is probably implicit, it was easy for us to take this (incorrectly) as being true of context #1 as well.

I think you're trying to say (1) above is incorrect? If yes, I have to disagree. Based on what John actually posted:

 

"So in summary,

all SFP+ modules will work fine in SFP cages as long as you don't mix speeds of the devices, both ends must be running at gigabit. If you use a switch with 10GbE ports that can be switched to 1GbE, they will work fine, but If the port ONLY runs at 10GbE, you cannot use it with a link where the other end is SFP." 

 

Pretty clear that if switch can be set for 1G link speed, which both CRS305 and CSS610 have settings for in their OS, then based on what John actually said, "they will work fine" (where "they" is referring to SFP+ modules). For whatever reason, neither of these Mikrotik switches do work as John describes. I don't understand why people want to keep trying to say they are working correctly. If this is correct operation, please explain the correct purpose of the 10/100/1G manual link speed setting if it's purpose is not to manually set link speed to 10/100/1G. 

 

[Jud]

1 hour ago, lasker98 said:

 

I think you're trying to say (1) above is incorrect? If yes, I have to disagree. Based on what John actually posted:

 

"So in summary,

all SFP+ modules will work fine in SFP cages as long as you don't mix speeds of the devices, both ends must be running at gigabit. If you use a switch with 10GbE ports that can be switched to 1GbE, they will work fine, but If the port ONLY runs at 10GbE, you cannot use it with a link where the other end is SFP." 

 

Pretty clear that if switch can be set for 1G link speed, which both CRS305 and CSS610 have settings for in their OS, then based on what John actually said, "they will work fine" (where "they" is referring to SFP+ modules). For whatever reason, neither of these Mikrotik switches do work as John describes. I don't understand why people want to keep trying to say they are working correctly. If this is correct operation, please explain the correct purpose of the 10/100/1G manual link speed setting if it's purpose is not to manually set link speed to 10/100/1G. 

 

Just trying to figure it all out, same as you. Your impression is certainly a fair interpretation of the quote.

 

But regardless, I would like to have a reasonably clear idea of how this works. (I believe that by next month I will have future proofed my home network for about the next 5 years or so, but you never know.)

[dbastin]

22 hours ago, jabbr said:

but part of switching a 10g port to 1g means using a dual rate 10g/1g module.

Firstly, I am not finding the compatibility table to help.

https://wiki.mikrotik.com/wiki/MikroTik_wired_interface_compatibility#10G_SFP+/25G_SFP28

 

In relation to SFP+ it doesn't refer to other manufacturers but refers to https://i.mt.lv/cdn/product_files/splitter_modules_180841.pdf

 

That document refers to (for example)  SFP+ CWDM module 10G SM 10km 1530nm Dual LC-connector DDM. 

 

Does the Dual mean dual rate?or Dual LC UPC?

 

The SFP+ modules listed in the document may all be discontinued.  Current SFP+

Modules = https://mikrotik.com/product/Splus85DLC03D (Units are tested and compatible with CCR1036-8G-2S+ and CCR1036-8G-2S+EM. Units are compatible with non-MikroTik SFP devices as well.)

 

AOC = https://mikrotik.com/product/s_ao0005

 

Note, both are dual rate 1G/10G, but no way of knowing which Mikrotik switch or router they are compatible with except the two mentioned.  And no way of knowing (from this info by Mikrotik) which non Mikrotik SFP+ modules are compatible with which Mikrotik switches and routers.

 

Am I correct assuming that not just ant dual rare 1G/10G module will work in any Mikrotik switch or router?

 

Is a Finisar or Cisco dual rate module likely to be better (spec, SQ) than Mikrotik modules?

 

For comparisin, from memory Ubiquiti community provides alot more more info about SFP compatibility.  Their range of SFP+ products seems more limited though ...

 

https://store.ui.com/collections/unifi-network-switching/products/switch-enterprise-8-poe

 

https://store.ui.com/collections/unifi-network-switching/products/switch-enterprise-8-poe

 

https://store.ui.com/collections/unifi-network-switching/products/us-xg-6poe

 

https://store.ui.com/collections/unifi-network-switching/products/unifi-switch-16-xg

 

And their DC voltage inputs are not terribly compatible with what psu we'd like to use.

[jabbr]

On 4/12/2023 at 3:54 PM, lasker98 said:

I'm assuming you're again referring to implementing this using dual rate SFP+? Will you confirm that you believe your Mikrotik switch (CRS305 or CSS610) is working correctly even though it doesn't respond to the auto negotiation or manual link speed settings in the switch OS when using non-dual rate SFP+ in your switch when connecting to your 1G endpoints?

 

I'm trying to be as specific as I can with my questions in hopes of getting an equally specific reply.

yes it’s working correctly. I don’t think any switch is required to support 1g with any module that doesn’t claim to support 1000base-X.

[jabbr]

9 hours ago, dbastin said:

Firstly, I am not finding the compatibility table to help.

https://wiki.mikrotik.com/wiki/MikroTik_wired_interface_compatibility#10G_SFP+/25G_SFP28

It’s a matrix … looks like the 305 switch which I have supports all the Mikrotik SFP and SFP+ modules listed

[jabbr]

9 hours ago, dbastin said:

And their DC voltage inputs are not terribly compatible with what psu we'd like to use.

I can’t get too excited about PSUs for Ethernet switches — the 10g and faster devices are tested to reject jitter and ground bounce regardless of PSU or at least with whatever ships with the device.

 

I personally keep the switches outside my audio area, and use a beefy iso transformer to supply my audio area. 

[jabbr]

19 hours ago, dbastin said:

And no way of knowing (from this info by Mikrotik) which non Mikrotik SFP+ modules are compatible with which Mikrotik switches and routers.

 

Am I correct assuming that not just ant dual rare 1G/10G module will work in any Mikrotik switch or router?

 

Is a Finisar or Cisco dual rate module likely to be better (spec, SQ) than Mikrotik modules?

The practice among switch vendors of professional grade equipment is that they provide branded and tested SFP(+) modules. 
 

The reason I’ve initially recommended Finisar is that few companies actually make optoelectronic transceivers and Finisar and Avago are among the few that are the OEMs. Cisco used to charge $1K for an SFP module which includes XXX for the Cisco brand name. 
 

The main purpose of this thread is so that people can report which 3rd party modules work and which don’t, that said it’s hardly a “bug” or error if a switch doesn’t support a module which it doesn’t claim to support. 
 

You have to realize that I recommended Mikrotik because it’s the most cost effective 10g and 100g switch (or at least was when I recommended) but the same issues apply to the Brocade switches I’ve used as well as the Mellanox 100g switch that I have ($33k list)

[89reksal]

12 hours ago, jabbr said:

yes it’s working correctly.

Thanks. I still haven't been able to get an explanation (after asking multiple times) of what the speed setting in the "Link" tab of the SwOS user interface, which has option to select from 10M/100M/1G, is supposed to do if it's purpose is not to set the link speed of the selected port to 10M, 100M or 1G. I have to assume God created that option for some reason. Hopefully you can finally answer this for me.

[Jud]

1 hour ago, lasker98 said:

I still haven't been able to get an explanation (after asking multiple times) of what the speed setting in the "Link" tab of the SwOS user interface, which has option to select from 10M/100M/1G, is supposed to do if it's purpose is not to set the link speed of the selected port to 10M, 100M or 1G

 

Someone who actually knows will give us the correct answer, but my impression, after reading and the experience I related, is "set the speed *if the ports and modules at each end of the connection permit it*." In other words, give you a GUI to choose from among various possibilities, not make the impossible possible. And yes, not tell you what's impossible. (Wonder if there's anything they could do about the last, but maybe they figure anyone playing with modules and manual speed settings ought to know. Shrug.)

[jabbr]

7 hours ago, lasker98 said:

Thanks. I still haven't been able to get an explanation (after asking multiple times) of what the speed setting in the "Link" tab of the SwOS user interface, which has option to select from 10M/100M/1G, is supposed to do if it's purpose is not to set the link speed of the selected port to 10M, 100M or 1G. I have to assume God created that option for some reason. Hopefully you can finally answer this for me.

I’m guessing that’s for the RJ-45 SFP because I don’t think any fiberoptic SFP modules support 10M or 100M … I’ve never used those speeds in the last 30 years or so. 
 

Unlike RJ-45 Ethernet who auto negotiates those speeds, fiberoptic doesn’t (necessarily) … I think most, if not all SFP28 ports/modules are multirate so consider stepping up to SFP28 if this functionality is critical — but 10M and 100M fiber is waaaaay legacy/unsupported these days. 

[Jud]

1 hour ago, jabbr said:

10M fiber...is waaaaay legacy/unsupported these days

 

Don't remember seeing the 10M option in my SwOS menu - @lasker98 what SwOS version?  Maybe I just missed it.

[jabbr]

Also out of habit and typically required, I set the link speed to what I want it to be. 

[Jud]

21 minutes ago, jabbr said:

Also out of habit and typically required, I set the link speed to what I want it to be. 

 

Within the range of what you  know the switch and module will allow (an assumption, but to assume otherwise would be ridiculous, I'm assuming 🙂).

[89reksal]

2 hours ago, Jud said:

what SwOS version?

2.13

 

2 hours ago, jabbr said:

I set the link speed to what I want it to be.

Ok, I'll bite; how do you do that since the speed select function doesn't work?

[Jud]

5 hours ago, Jud said:

 

Don't remember seeing the 10M option in my SwOS menu - @lasker98 what SwOS version?  Maybe I just missed it.

 

Yep, missed it - my eyes saw it, but my brain just said "10G," since I haven't ever dealt with a 10M optical connection.

[jabbr]

On 4/14/2023 at 9:10 PM, lasker98 said:

Ok , I'll bite; how do you do that since the speed select function doesn't work?

 Again the SFP module has to both be supported by the switch and support the selected speed. It’s data sheet states which speeds it supports. Are you using a dual rate module? 

[jabbr]

On 4/14/2023 at 7:23 PM, Jud said:

 

Within the range of what you  know the switch and module will allow (an assumption, but to assume otherwise would be ridiculous, I'm assuming 🙂).

Absolutely. The port needs to be set to a speed explicitly supported by the SFP(+) module.

 

When in doubt the manufacturer user manual may supply additional details. 
 

my Mellanox switch which has a command line, knows quite a bit about each module indicating that there is active communication between the switch and the modules. It’s not nearly as simple as some have portrayed.

[R1200CL]

2 hours ago, jabbr said:

Again the SFP module has to both be supported by the switch

This more or less only applies to Cisco switches. And even there my Finisar module works. 
 

So I’m not agreeing with this argument. And we know for sure Finisar is working with MikroTik. 
 

Here is a list of several posts that support what I’m saying. 
https://audiophilestyle.com/search/?&q=Sfp&quick=1&author=JohnSwenson&search_and_or=and&sortby=relevancy

 

Here is a bit more about the subject:

http://www.toad.com/gnu/sysadmin/sfp-lockin.html

 

https://community.fs.com/news/compatibility-guarantee-for-fs-transceivers.html

 

 

[Jud]

1 hour ago, jabbr said:

knows quite a bit about each module

What it "knows" has got to be limited to what the electronics inside the module can communicate (plus possibly what algorithms in the switch, which may not exist in cheaper switches, can make of that). What sort of information are we talking about?

[Jud]

2 hours ago, R1200CL said:

This more or less only applies to Cisco switches. And even there my Finisar module works. 
 

So I’m not agreeing with this argument. And we know for sure Finisar is working with MikroTik. 
 

Here is a list of several posts that support what I’m saying. 
https://audiophilestyle.com/search/?&q=Sfp&quick=1&author=JohnSwenson&search_and_or=and&sortby=relevancy

 

Here is a bit more about the subject:

http://www.toad.com/gnu/sysadmin/sfp-lockin.html

 

https://community.fs.com/news/compatibility-guarantee-for-fs-transceivers.html

 

 

 

You realize neither the article you cited about vendor lock-in nor the very first post under the John Swenson search support what you are saying?

[89reksal]

4 hours ago, jabbr said:

Are you using a dual rate module?

In my Mikrotik CRS305 I had two Finisar FTLX1475D3BCV dual rate modules working for two days last week. I purchased these new from Mouser. I also purchased two new FTLX1475D3BCL SFP+ modules at the same time. I purchased these mainly based on your and others experience and comments that these dual rate modules were able to be used in this switch as a confirmed configuration to have an SFP+ module operating at 1G. 

 

Even these were not simple to get working initially. These replaced two Finisar FTLF1318P3BTL SFP modules that were operating at 1G with "auto negotiation" selected in SwOS ver. 2.13. On the other end of these two links were two FTLX1471D3BCL SFP+ modules, one installed in my ER, the second installed in my Optical Module Deluxe. This configuration was rock solid. It always consistently worked after all failed attempts trying different SFP+ modules in the Mikrotik. It was just plug and play.

 

Getting back to this part, "not simple to get working initially". Based on previous comments made about the 1475 dual rate modules, I fully expected to simply exchange the 1318 modules for the new 1475 dual rate modules, still with "auto negotiation" selected. Nothing, no link established. The new modules were correctly recognized and identified in the SwOS "SFP" tab. I tried manually setting the link speed to 1G and still nothing. All these attempts also involved various combinations of power downs, reboots, etc., but nothing. I thought maybe it was because I was still using the 1471 modules at the other end. I replaced these with the new FTLX1475D3BCL modules, still nothing (again after trying multiple power down, reboot, etc. combinations).

 

Back at the Mikrotik end, after multiple attempts, even trying to set up in RouterOS, I somehow got 1G links established after trying all the different manual link speed options back in SwOS. The last thing I did was manually set 100M, then set "auto negotiation". Everything worked. This all took well over an hour and about $500.00 (Canadian) for the four new modules. Not thrilled but at least I finally had a 10G switch working with SFP+ modules.

 

Two days later I went to play music and nothing. After about an hour of trying everything I had tried to get it working originally, I went back to the 1318 modules and again, these instantly worked. As a side note, thank God for the Net Analyzer app. I can't imagine going through all this without it.

 

So to finally answer your original question;

4 hours ago, jabbr said:

Are you using a dual rate module?

no. If someone is able to tell me how to get them working in my switch at 1G it would be much appreciated.