What better for music storage ?

[Roma_V]

Hi ,What do you think it's better to storage music external or internal HDD ? External via Ethernet or USB3 ? Now i have something like 3TB + of music , and i cannot storage in ssd disk .... Best regards ,Roman .

[astrotoy]

If you are going to have a growing collection, then adding drives outside your computer are much more convenient than changing internal drives inside your computer. I have both external USB and NAS drives which have internal drives which are connected by Ethernet. Both work fine for music files (hirez or regular). The external drives are cheaper. I buy 4TB external drives from Costco for $140. I have settled on Western Digital Red Drives for my NAS. They work at a slower speed (doesn't matter for music files) and cooler temperature. They cost more than external drives, plus you need to pay for the NAS itself.

 

You can more easily get some redundancy with a NAS - RAID 5 or 6 (I use the Synology RAID system which allows me to mix drive sizes), but it is not as safe as a complete 1:1 backup, which you can do with an external drive fairly easily.

 

Currently, I have over 24TB of music files (the vast majority ripped from vinyl and prerecorded tapes by me at 192/24). It has taken about 4 years so far to do it so back ups for me are essential (I have four levels of backup!).

 

With 3TB and growing, I would say that at least one level of back up is essential - two, including one at a neighbor's house is better.

 

Larry

[sandyk]

Bear in mind that there are now quite a few reports about the very large capacity HDDs not sounding as good as the smaller ones. Possibly due to increased short term current demands with the extra platters?

[One and a half]

I have given up on external drives, tried Seagate, WD, Fantom for the last 4 years, all of them with the same problem eventually. If they are powered up 24/7 even with spin down (PITA anyway), the heat is trapped by the enclosure and doesn't dissipate properly. Even the Fantom drive enclosures are made of aluminium (AL), the heat is still retained and the controllers inside the portable drives fail.

 

The HDDs themselves survive quite well and can be reused. 3.5in drives need airflow, and then they are happy. Either in a NAS or a computer housing to allow for breathing space around the drive with air movement from fans (no PWM required).

2.5in drives have a lower heat dissipation, but don't have the capacity. More drives are needed to maintain the capacity, roughly these days 3:1 to a 3.5in drive. Means more heat again most likely back oto a 3.5in drive anyway, plus controllers and either separate eSATA , SAS enclosures.

 

The only application where a portable drive works is for removable backup purposes, where the drive is connected, does it's job and is powered off completely for some time allowing it to cool.

 

That's before we consider which drives are optimum for audio.

[Allan F]

Hi ,What do you think it's better to storage music external or internal HDD ? External via Ethernet or USB3 ? Now i have something like 3TB + of music , and i cannot storage in ssd disk .... Best regards ,Roman .

 

I would also recommend external storage. A Network Attached Storage (NAS) in a RAID configuration is a good option. RAID involves writing the same data to two drives simultaneously, providing redundancy in case one of the drives fails. As astrotoy has pointed out, 'redundancy' is not the same as backup, which you also need to provide for. A NAS on a network allows you to share your music library among several devices. For example, I access my music library from both my main stereo system in the living room with its music server, as well as my desktop computer in my bedroom.

 

You can also store your music on external drives attached to your music server computer. However, if the music server computer is connected to a USB DAC, you don't want to store music on an external USB drive attached to the same computer. It will create timing problems because of competition for use of the USB bus. In that situation, an external drive can be connected by other means such as SATA or Thunderbolt.

 

A music server with a Solid State Drive (SDD) for Operating System and program storage is likely to generate less 'noise' than an internal spinning hard drive. Another reason for opting for external music storage. At present, the high cost of SDDs makes them impractical for music storage.

 

Regardless of how you store your music library, backup to external storage which can be disconnected and safely stored after backup creation is essential.

[sandyk]

Personally, I think that an internal SATA SSD using an improved PSU and screened minimum length SATA3 6GB/s leads would be the way to go, but presently they are too costly in the larger storage sizes to be a realistic option.

As others have shown elsewhere already, a simple additional low noise, +12V from the internal SMPS to a low noise +5V supply for a SATA drive shouldn't take up too much internal space either, as SSD current requirements are quite modest.

Doing this also helps to prevent SSD current variations from being reflected back into other areas of the computer/server.

As far as I am concerned, the K.I.S.S. method is normally best.

Less frequently played material could be saved externally and accessed via Firewire for example.

 

Alex

[Roma_V]

Unfortunately as I thought all turns difficult . I understand that it is best to use a SSD HDD but I need 4TB HDD , and it's too expensive . Do you think there is a difference between WD HDD RED and GREEN ? Maybe better to buy another company ? I've seen bad reviews about WD GREEN . Thank you "Allan F" , I think you're right , it is better not to use USB storage if using USB DAC's .

 

Best regards ,

Roman .

[tgb]

hi Roman,

I have a Syno 212J, 2×2To WD Caviar Green rather old but working perfectly.

In RAID0 because a NAS is made to make the files available easily but not to back up ;-). In Raid0 access to data is quicker, the nas is dead cool.

Then of course I have back up on standard external Hdd using usb. Back up is doubled in fact ;-) I bought yesterday a WD external 3to usb3.0 at 0.033Eur/Go... Back up becomes a non-pricing issue nowadays.

This is my setup.

Rgds

[Allan F]

Do you think there is a difference between WD HDD RED and GREEN ? Maybe better to buy another company ? I've seen bad reviews about WD GREEN . Thank you "Allan F" , I think you're right , it is better not to use USB storage if using USB DAC's.

 

You are welcome, Roman. FWIW, WD Red Drives are specifically designed to work in a NAS configuration and are likely a better choice for that purpose. While I don't know the technical details, it is my understanding that the firmware of the Red drives differs from that of the Green and, among other things, contributes to greater reliability .

[Roma_V]

In the construction of the audio system is always a compromise ... I'll buy the WD Red internal .

[keeper]

Once you get to this level of storage requirement wouldn't streaming be the better option and let someone else worry about storage?

[k6davis]

I would also recommend external storage. A Network Attached Storage (NAS) in a RAID configuration is a good option. RAID involves writing the same data to two drives simultaneously, providing redundancy in case one of the drives fails. As astrotoy has pointed out, 'redundancy' is not the same as backup, which you also need to provide for.

 

I'm really at a loss about the benefits of RAID redundancy for music storage.

 

I've got 16TB of capacity in my library. Buying a duplicate set of drives that will get the same amount of wear as the primary set and that will still not provide backup seems to be both high cost and low benefit. RAID Redundancy seems like a great idea for an operation that needs to maximize uptime. If Amazon is down for 10 minutes, they lose millions. If I lose a drive in my system, I can wait for a little while, while my library is rebuilt from the backup.

 

If I'm going to spend the money to store 3 copies of my library, I'd rather have 1 main, 1 onsite backup and 1 offsite backup than 2 copies in the RAID and 1 real backup.

[Audio_ELF]

I'm really at a loss about the benefits of RAID redundancy for music storage.

The benefits of RAID for music storage usually come when you are talking about striping (RAID5) rather than mirroring.

 

If you have 16TB; this can be stored as 4 x 4TB (single drives or mirrored pairs). These will then appear as 4 individual drives.

 

Alternatively if you use RAID5; 5 x 4TB drives combined give you the same 16TB storage, but will appear as a single 16TB drive making file management easier.

 

Eloise

[Smitty77_7]

Once you get to this level of storage requirement wouldn't streaming be the better option and let someone else worry about storage?

 

Is 'cloud storage' as an additional 'off site' backup an option? Tagged onto the streaming comment, as it was close to my question.

 

Smitty

[wwaldmanfan]

I have given up on external drives, tried Seagate, WD, Fantom for the last 4 years, all of them with the same problem eventually. If they are powered up 24/7 even with spin down (PITA anyway), the heat is trapped by the enclosure and doesn't dissipate properly. Even the Fantom drive enclosures are made of aluminium (AL), the heat is still retained and the controllers inside the portable drives fail.

 

The HDDs themselves survive quite well and can be reused. 3.5in drives need airflow, and then they are happy. Either in a NAS or a computer housing to allow for breathing space around the drive with air movement from fans (no PWM required).

2.5in drives have a lower heat dissipation, but don't have the capacity. More drives are needed to maintain the capacity, roughly these days 3:1 to a 3.5in drive. Means more heat again most likely back oto a 3.5in drive anyway, plus controllers and either separate eSATA , SAS enclosures.

 

The only application where a portable drive works is for removable backup purposes, where the drive is connected, does it's job and is powered off completely for some time allowing it to cool.

 

That's before we consider which drives are optimum for audio.

 

I am happy with this product from Other World Computing, a bus-powered FW800/USB3, dual mini 2.5" RAID HDD, available in several speed/capacity configurations up to 4TB. I have the 2TB 7200 rpm model, configured in RAID 0 (striped mode). Read/write times over FW800 are very fast, and there is no internal fan or external SMPS to generate noise. This is the primary drive for my music server. It spins down when I put my MBP to sleep after a listening session.

 

https://eshop.macsales.com/shop/firewire/EliteALmini/RAID/eSATA_FW800_FW400_USB

 

I also have three fanless 1TB G-Technology 2.5" external mini HDD's, also bus-powered from FW800 or USB3. These are my work/safety backup drives. In two years of constant use, I have had a controller failure on one of them, but the entire unit was quickly replaced under the mfg's three year warranty.

[jabbr]

I'm really at a loss about the benefits of RAID redundancy for music storage.

 

All depends on what you need to do.

I used mirrors frequently. Advantages:

1) prevents bit rot

2) improves access speed

3) seamless recovery if a drive faults

 

I've got 16TB of capacity in my library. Buying a duplicate set of drives that will get the same amount of wear as the primary set and that will still not provide backup seems to be both high cost and low benefit. RAID Redundancy seems like a great idea for an operation that needs to maximize uptime. If Amazon is down for 10 minutes, they lose millions. If I lose a drive in my system, I can wait for a little while, while my library is rebuilt from the backup.

 

Depends on where you want to allocate your funds and time. Maintaining a 16TB backup set is a hassle. Do you worry that one of your backup drives will either fail or blow an important sector. In that case you may unknowingly introduce errors during your recovery. Been there done that...

 

My family photos and videos are worth real data protection and backup efforts. The audio library just goes along for the ride.

[jabbr]

The benefits of RAID for music storage usually come when you are talking about striping (RAID5) rather than mirroring.

 

If you have 16TB; this can be stored as 4 x 4TB (single drives or mirrored pairs). These will then appear as 4 individual drives.

 

Alternatively if you use RAID5; 5 x 4TB drives combined give you the same 16TB storage, but will appear as a single 16TB drive making file management easier.

 

Eloise

 

I use ZFS which allows any of the options.

 

The problem with 4x4 pure striped RAID is that there's no redundancy. A single drive failure takes out the entire 16TB. Since there are four drives, the chance of a drive failure is 4x that of a single drive.

RAID5 is minimally acceptable but risks a second drive failing during the time it takes to resilver the array.

RAID6 minimizes this by having 2 redundant drives. So ... assuming 4 total drives, striped = 16gb, raid5 = 12 gb, raid6 = 8 gb.

 

Now suppose you mirror and stripe (RAID10) ... you get the same 8 gb. That's what I do. You can also reasonably easily enlarge the array by adding an additional stripe.

 

The point is that the hardware and software allows any of the options. Saving a few pence while giving up redundancy is short sighted IMHO.

[jabbr]

Bear in mind that there are now quite a few reports about the very large capacity HDDs not sounding as good as the smaller ones. Possibly due to increased short term current demands with the extra platters?

 

Ahh there are so many variables. Actually for a given dataset size, large HDDs use less current. That said hardware is often designed to stagger spin-up. However, the higher the number of drives, the more spin-ups that occur. So lots of small drives might actually be worse than fewer large drives.

 

Likewise vibration. It had been well demonstrated that inadequately mechanically dampened drives have a much higher error rate. All this is easily measured. The data integrity algorithms in modern systems handle these issues.

 

You may wish to investigate the audio "performance" of regular vs. SED drives however:-)

[Paul R]

I'm really at a loss about the benefits of RAID redundancy for music storage.

 

I've got 16TB of capacity in my library. Buying a duplicate set of drives that will get the same amount of wear as the primary set and that will still not provide backup seems to be both high cost and low benefit. RAID Redundancy seems like a great idea for an operation that needs to maximize uptime. If Amazon is down for 10 minutes, they lose millions. If I lose a drive in my system, I can wait for a little while, while my library is rebuilt from the backup.

 

If I'm going to spend the money to store 3 copies of my library, I'd rather have 1 main, 1 onsite backup and 1 offsite backup than 2 copies in the RAID and 1 real backup.

 

And you are exactly right. RAID in a redundant Mode 0 is not something I would ever advise for a music system. RAID5 or better however, *is*. Reading form a raid 5 unit is faster than from a single disk, and if a disk fails, you don't stop operating. You just stick in a new disk and it rebuilds itself with no further intervention.

 

However, you do have to have a set of *backup disks* then, which don't have to be raid at all. Just offline copies of all your music.

 

Real world case: when I first started playing with JRiver, I had a setting wrong and it went into my music files and hosed them up pretty good, rewriting meta data and all that. Since the real data was on a backup, all I had to do was restore the original media files from backup and try again. That taught me really quickly that it was possible to hose up one's entire music library (media) files in an instant. RAID5 was being used in this case, but of course, it merely propagated the unwanted changes.

 

Does that make sense?

 

-Paul

[wwaldmanfan]

RAID in a redundant Mode 0 is not something I would ever advise for a music system. RAID5 or better however, *is*. Reading form a raid 5 unit is faster than from a single disk, and if a disk fails, you don't stop operating. You just stick in a new disk and it rebuilds itself with no further intervention.

 

However, you do have to have a set of *backup disks* then, which don't have to be raid at all. Just offline copies of all your music

From Wikipedia: "A RAID 0 (also known as a stripe set or striped volume) splits data evenly across two or more disks, without parity information and with speed as the intended goal. RAID 0 was not one of the original RAID levels and provides no data redundancy. RAID 0 is normally used to increase performance, although it can also be used as a way to create a large logical disk out of two or more physical ones."

 

My setup yields a portable, fanless, silent, 2TB FireWire-bus-powered HDD that reads/writes at very high speed. This OWC product is available up to 4TB, albeit at a slightly lower operating speed (5400 rpm vs. 7200 rpm). I'm not aware of another off-the-shelf unit like it on the market.

My goal was not RAID data redundancy. I have *three* other external HDD's, each with mirrored backups of my entire music library, and all updated regularly.

[k6davis]

Thanks to everyone for your replies. I have a better understanding of the benefits now.

 

It would definitely be more convenient to be able to just slip a replacement drive in to replace a faulty one and have the system automatically rebuild the volume. As it stands, I'd have to wait for my entire library to copy over from the backups. But something about having (8) 4TB drives spinning at all times just rubs me the wrong way.

 

I'm using Windows Storage Spaces to create a single 16TB volume from (4) 4TB drives in both my active library and in my backup library and it has worked flawlessly for me so far. Maintaining the backup is a snap with differential backup software - I use the excellent and very comprehensive shareware DirSyncPro.

 

When the 16TB gets cheaper, then I'll consider a RAID, but for now, I feel that it isn't the best allocation of my resources.

[Paul R]

From Wikipedia: "A RAID 0 (also known as a stripe set or striped volume) splits data evenly across two or more disks, without parity information and with speed as the intended goal. RAID 0 was not one of the original RAID levels and provides no data redundancy. RAID 0 is normally used to increase performance, although it can also be used as a way to create a large logical disk out of two or more physical ones."

 

My setup yields a portable, fanless, silent, 2TB FireWire-bus-powered HDD that reads/writes at very high speed. This OWC product is available up to 4TB, albeit at a slightly lower operating speed (5400 rpm vs. 7200 rpm). I'm not aware of another off-the-shelf unit like it on the market.

My goal was not RAID data redundancy. I have *three* other external HDD's, each with mirrored backups of my entire music library, and all updated regularly.

 

Think about that for a moment, in RAID 0 (software or hardware raid) if you loose one drive, you risk an entire data partition. Or to put that another way, you risk all your files, not just the ones on a particular volume. Not worth it!

 

Raid 1 will cut your storage space in half. Raid 5 uses about 20% of your storage space for redundancy and parity information. Raid 6-10 use different and varying amounts.

 

You really are not gaining ground until you hit RAID-5 IMNSHO.*

 

-Yours,

-Paul

 

* With one exception. If you are using a SAN storage system, it may make perfectly good sense to combine SAN volumes in RAID-9 striping. The actual SAN volumes are usually physically implemented as RAID-5 or better anyways. Combining two or more SAN volumes is perfectly safe. I do that sometimes to make multiple terabyte volumes available that I can expand easily. Not at all common on home music libraries though!

 

-Paul

[wwaldmanfan]

Think about that for a moment, in RAID 0 (software or hardware raid) if you loose one drive, you risk an entire data partition. Or to put that another way, you risk all your files, not just the ones on a particular volume. Not worth it!

 

I'm not too worried. If my RAID 0 drive goes down, I have three other identical external HDD copies of my entire music library, and the RAID gets sent back to the mfg. for warranty replacement (3 years). In the meantime, I have 2TB of storage in a silent, portable FW800/USB 3 unit powered from my laptop's internal battery, with no AC required.

[Paul R]

I'm not too worried. If my RAID 0 drive goes down, I have three other identical external HDD copies of my entire music library, and the RAID gets sent back to the mfg. for warranty replacement (3 years). In the meantime, I have 2TB of storage in a silent, portable FW800/USB 3 unit powered from my laptop's internal battery, with no AC required.

 

I am not worried either, as I have multiple copies of everything. Versioned and all that jazz too.

 

But I am not sure the OP is in the same enviable position we are.

 

-Paul

[occamsrazor]

All depends on what you need to do.

I used mirrors frequently. Advantages:

1) prevents bit rot

 

As far as I'm aware, unless you're running a file system that performs checksums like ZFS, then running some kind of Raid mirror isn't going to help against bitrot. But am happy to be corrected...