Best way to add multiple HDD connections to a motherboard?

[AudioDoctor]

I am heading full steam ahead in building a machine which has as its sole purpose to serve and store music files for playback on my network. This machine will be a desktop machine with a bunch of HDDs in some ZFS set up yet to be determined. The OS will be Ubuntu server LTS. I want to use an ASUS X670e pro art motherboard. This motherboard has only 4 SATA ports which is why I am exploring PCIe cards that add more SATA ports. I am far outside my knowledge base here and have no idea what is good, what I am looking at, what I should avoid, etc... I have learned one thing so far and that is to avoid any card with an onboard RAID controller as ZFS will be taking care of that.

 

Any suggestions?

[AudioDoctor]

4 hours ago, davide256 said:

Can't say I have any expertise but this article may be food for thought. Author definitely dislikes  cheap SATA expansion cards.

 

https://unraid-guides.com/2020/12/07/dont-ever-use-cheap-pci-e-sata-expansion-cards-with-unraid/

 

 

Thanks, that's informative!

[AudioDoctor]

SO basically, I need this, unless I want to figure out how to flash one.

https://www.pc-pitstop.com/9305-24i

[AudioDoctor]

27 minutes ago, Jud said:

 

I don't know enough about TrueNAS to understand why you don't need to flash this. (BTW, I see a 12Gbps 9500 series for only $140 more. 😉)

 

This one does not include a raid controller, according to the first link shared by David above, and therefore does not need to be flashed into a different mode to turn that function off. 

 

Second, I am not going to be using TrueNAS, rather Ubuntu Server. TrueNAS is incapable of running Minimserver. I am keeping my TrueNAS device as pure backup, local and offsite.

[AudioDoctor]

This monster will be going into a Define 7 XL case which can fit up to 18HDDs. I am planning on using 12 which leaves me 4 spare power ports available for future expansion.

 

edit: I will also be using ECC RAM, undecided how much though, probably 64GB.

[AudioDoctor]

13 minutes ago, yamamoto2002 said:

For 18 drives, I recommend SAS Expander enclosure like Netstor NS385S-8028

and any SAS host bus adapter with external SAS connector. I'm long term fan of Adaptec so I'll choose something like HBA Ultra 1200-16e. While many people prefer second hand Broadcom/Avago/LSI cards

 

Or goes to Gamers Nexus route, Norco RPC-4224 and two SAS HBA with internal connector fan out cable and direct attach

 

 

I wasn't aware that something like this existed, thanks.  This is exactly why I asked. However, other than more drive space expandability, what benefit would this give me over putting 12HDDs in the computer case?

[AudioDoctor]

2 minutes ago, Jud said:

Unable to come up with pricing on the Netstor products.

 

Same, I can't either.

[AudioDoctor]

Just now, Jud said:

TrueNAS docs re SAS expanders:

 

"While SAS expanders, designed for SAS disks, can often support SATA disks via the SATA Tunneling Protocol or STP, we still prefer SAS disks for reasons mentioned in the NL-SAS section above (SATA disks function on a SAS-based backplane)."

 

IOW, in my extremely limited understanding, "We folks at TrueNAS like HBAs better than SAS expanders for [reasons]." 

 

Yep, I didn't forget you're not building a TrueNAS machine.  Just figuring these folks have a lot of experience with issues similar to those you're working through.

 

I think you're probably right Jud.

[AudioDoctor]

1 minute ago, Jud said:

 

I think I may have gotten it backwards, hard for me to tell. 🙂 They're saying they prefer SAS disks rather than SATA disks for SAS expanders, I believe.

 

While it's an interesting idea, I think it's beyond what I want to do anyway. The one box solution I have is in my opinion a better idea and less complex.

[AudioDoctor]

8 minutes ago, Jud said:

10TB SAS drive on Amazon (just for completeness, @AudioDoctor has already mentioned this is probably beyond the scope of what he wants to do):

 

https://www.amazon.com/Ultrastar-HUH721010AL5200-0F27352-Platform-Enterprise/dp/B01GXUR98G?th=1

 

Thanks Jud, I will look that drive up elsewhere as I intentionally block Amazon and eBay on my network.

[AudioDoctor]

For comparison, this is what I am likely using:  https://www.bhphotovideo.com/c/product/1731376-REG/wd_wdbc9v0060hh1_wrsn_6tb_red_plus_cmr.html

[AudioDoctor]

11 minutes ago, Jud said:

 

Another SAS drive example, this one from Newegg:

 

https://www.newegg.com/toshiba-mg06sca800e-8tb/p/1Z4-000B-009N6?Item=1Z4-000B-009N6&Description=sas drives&cm_re=sas_drives-_-1Z4-000B-009N6-_-Product

 

Thanks. I should block Newegg too. That is nearly double the price of my intended drive, not to mention other increased costs to accommodate it and it's just too much of everything for my use.

[AudioDoctor]

9 minutes ago, Jud said:

 

You can block the objectionable stuff from these sites with the Vivaldi browser and Privacy Badger. Kind of having your cake and eating it too.  Unless of course you find them objectionable in their totality. 🙂

 

Newegg sold me a fake power supply from china when I ordered a real one from the USA instead. They then refused to do anything about it. So I no longer deal with them. 

 

I object to Amazon on moral grounds and refuse to do business with them and eBay is just asking to be scammed, imo.

 

I already run AdGuard Home on my Netgate device running PFSense software. It blocks all kinds of things, and now Newegg as well.

[AudioDoctor]

On the other hand, Microcenter has never done me wrong. Neither has B&H Photo and Video.

[AudioDoctor]

6 minutes ago, Jud said:

Me neither. OTOH, I‘ve purchased most of the components for several computer builds from Newegg and been very happy.

 

Until that incident, I had not had a problem with Newegg either. That they refused to do anything about it was the final nail in the coffin.

[AudioDoctor]

Or each cable splits into 8 SATA drives and I put them in the case normally...

[AudioDoctor]

In my research I found this card and this cable are operable together

 

https://www.pc-pitstop.com/broadcom-hba-9500-16i-tri-mode-storage-adapter-05-50077-02-2

 

https://www.diliving.com/SlimSAS-X8-to-8-SATA-SFF-8654-74P-to-8-SATA-7P-SlimSAS-Host-RAID-to-SATA-Target-HD-85CM-Cable-p3247776.html

 

There are also cables that break out to 4 SATA drives with the same slimSAS connector at the other end.

[AudioDoctor]

19 minutes ago, yamamoto2002 said:

The Broadcom card on the picture mates 4 SFF-8643 connectors and it is Mini SAS HD and it is not SlimSAS

 

Edit: It seems the picture shows different product and 9500-16i has SlimSAS 8i connector...

 

 

 

edit: It is good to know that the picture is not what is listed in the specs, thanks.

[AudioDoctor]

Or this cable from the same vendor:  https://www.pc-pitstop.com/1m-sff-8654-to-8-x-sata-cable-without-sideband

 

[AudioDoctor]

5 minutes ago, yamamoto2002 said:

Yeah, sideband connector goes to nowhere so it is better without it

 

Edit: both cables do not have sideband cable, so difference is the length, I think cheaper one is better

 

The cables look identical to me.

 

$18.88  https://www.diliving.com/SlimSAS-X8-to-8-SATA-SFF-8654-74P-to-8-SATA-7P-SlimSAS-Host-RAID-to-SATA-Target-HD-85CM-Cable-p3247776.html

 

$79.99  https://www.pc-pitstop.com/1m-sff-8654-to-8-x-sata-cable-without-sideband

[AudioDoctor]

1 hour ago, Jud said:

 

I think the way you intend to do it will be much less aggravation for you, but just in case anyone is ever researching this topic again, I ran across this thread regarding a tutorial from the Minimserver developer on the Minimserver forums (linked the last page from late 2022 so we know the information is fairly current): https://forum.minimserver.com/showthread.php?tid=2891&page=13

 

That's what made me not want to try and run Minimserver on TrueNAS.

[AudioDoctor]

Plans are changing due to the difficulty of tracking down DDR5 ECC RAM right now. Looks like i'll step down to a 5600 or 5700x and DDR4 ECC ram, a different motherboard. This means I'll need a separate, cheap, gpu as well as a 10G card.

 

If I plug an NVME into a different computer, install Ubuntu Server on it, and then put it in the machine and boot it, will it work with no GPU and allow me to SSH into it as I normally would to administer it?

[AudioDoctor]

1 hour ago, jabbr said:

That is exactly what I do except that with NVME drives you want this adapter: https://www.pc-pitstop.com/broadcom-p411w-32p-x16-pcie-nvme-switch-adapter-board-05-50054-00

 

and go for the ConnectX-4 series (25G) because ConnectX-3 isn't still supported: https://www.fs.com/products/119650.html

 

Is there a reason I can't plug the NVME into an open slot on the motherboard in my PC and do it that way?

 

Does the Mellanox card operate at 10G as well the 25G?

[AudioDoctor]

2 hours ago, Jud said:

 

If you have sshd running on the Ubuntu Server you have installed (I haven't used Server, guessing it comes with sshd included), and if the NVMe/Server is indeed the drive/OS you're booting from (as you'd ensure from the BIOS/EFI), then I see no reason why it shouldn't work.  Why don't you try it out as a test with a spare drive if possible, not only to be sure it works, but to get accustomed to the drill?

 

When installing server it gives the option to install SSH and set it up so it works when the machine boots.

[AudioDoctor]

1 hour ago, AudioDoctor said:

 

Is there a reason I can't plug the NVME into an open slot on the motherboard in my PC and do it that way?

 

Does the Mellanox card operate at 10G as well the 25G?

 

Quoting myself here, I saw the specs for the card and yes it does indeed run at 10G. However, does a 25GbE SFP slot require a different transceiver than a 10GbE or are they the same? Will a 10GbE transceiver work in the 25GbE slot? If I need identical transceivers at both ends, will the 25GbE transceiver work at the 10GbE switch sip slot?

 

I am truly out beyond all my previously acquired knowledge here.