Jump to content
  • The Computer Audiophile

    Reevaluating My Music Storage - Update

     

     

    Audio: Listen to this article.

     

     

    A couple weeks ago I wrote about reevaluating my music storage. My backup NAS died, leaving me with a single QNAP TV-872XT for music storage. The more I researched possible solutions, and read the great comments from members of this community, the more mentally paralyzed I got. I set the topic aside for several days, then took a look at it with fresh eyes. I've now settled on a solution that works for me. Here's the solution, and more importantly why I selected it. 

     

    I've elected to keep my QNAP TVS-872XT as the main music storage device. This NAS is extremely capable, and remains a current model in the QNAP lineup after many years. To speed up, or at least make sure the files stored on my QNAP are served as fast as possible, I will add four additional 6TB drives to the unit. This will fill all eight drive bays. Then the eight drives will be configured as a ZFS RAID0 array. Huh? Isn't this an audio site? I know, sometimes it can get geeky. 

     

    A vdev (virtual device) or multiple vdevs, make up a storage pool, which contains the folders that store our music. A simple way to look at this is, physical disk(s) > vdev > pool > folders. Don't ever use that as an answer on a test, but for our purposes of storing and playing music, it's all we need. 

     

    On a ZFS based QNAP NAS, running the QuTS operating system rather than the QTS operating system, configuring the array as RAID0 places a single vdev on each physical disk. The more vdevs striped together, the faster the storage pool becomes. In my case, I'll have eight vdevs because I'll have eight drives. I'm using 7200 RPM Seagate Ironwolf NAS drives, so this NAS will deliver files as fast as possible. I talked with QNAP about adding an NVMe drive as cache, but I was told this would not increase the speed because a RAID0 array is so fast. 

     

    I'm currently using four drives in a ZFS RAID0 array, and as soon as my additional four drives arrive, I'll start from scratch with a new eight drive array.

     

    A large RAID0 array like this can saturate a 1Gbps network link very easily. It wouldn't make much sense for me to use this configuration without a 10Gbps network. I've added a 10Gb interface to my MacBook Pro (much more on this later), and connected it to my exiting 10Gb network, and connected the QNAP's 10Gb interface to the same 10Gb switch. My MacBook Pro still uses the 1Gb Thunderbolt interface as well, for my Merging Ravenna hardware. 

     

    In my testing with only four drives, I can play 12 channel 24/352.8 6GB files without an issue on my MacBook Pro. The files load very fast. 

     

     

    HOLD UP, RAID0 IS DANGEROUS!

     

    If one drive goes bad, I lose all my data! Yes, that's true. To counter this "issue" I've added 16TB USB drives to the QNAP, and created Hybrid Backup Sync job that runs automatically, to mirror the data on my NAS to the USB drives. If a drive in my RAID0 array goes bad, I will replace it, then reverse the synchronization to restore the data. I'm not running a datacenter where the uptime must be 24/7/365. If my local music files are unavailable for a couple days, while the restore is running, I'm OK with that. 

     

    Adding suspenders to my USB backup belt, I installed iDrive directly on my QNAP, and am backing up to the cloud right now. In my tests, uploading and restoring from the cloud is very fast. I certainly won't use this as my first restore option, but it's there if I need it. The cost for iDrive is very reasonable (pricing). The first year of 20TB of cloud storage cost me $20. Additional years will cost $200.

     

    Note to people interested in iDrive. I recommend using your own encryption key as part of your account setup. This keeps your data completely encrypted and unavailable to even people at iDrive. Using the default encryption key won't keep iDrive from decrypting your data. Using a private key is as easy as specifying a string of characters that you make up, like a password, as part of the setup. 

     

     

    Local Storage?

     

    I very seriously considered using local storage on my MacBook Pro. The cost of internal storage was quite high, at around $2,400 for 8TB. This still wouldn't hold my entire library, so I'd have to split it up. An external 8TB NVMe drive is about $1,300, but then I have something hanging off my laptop. A larger NVMe array is much more money and much larger. Either way, I still need to backup this data, then make the backup available for other computers I use for music playback. NVMe is certainly fast and easy, in some situations, but I just don't think it's the right solution for my use case. 

     

     

    Best Solution For My Use Case

     

    A centralized NAS, with automatic backups, that's available to all computers on my network, and is fast enough to saturate a 10Gb Ethernet network, is right for me. 

     

    Adding the 10Gb Ethernet to my MacBook Pro was both easy and elegant. I will write about this next week. I absolutely love what I'm using and thin kit deserves its own article. 

     

    I likely missed some reasoning above, as my brain worked overtime considering all the options. I'm happy to discuss more reason for why I selected this storage solution. The why is much more important the the what. 

     

     




    User Feedback

    Recommended Comments



    20 minutes ago, bobfa said:

    Chris did you see anything in the logs for Hybrid Backup???  This is scary.  I know a lot of folks that use that tool!

     

    NOTHING! 

     

    Check this out. I ran a test last night on one folder to make sure I wasn't losing my mind. It says no files skipped. 

     

    Screenshot 2023-02-10 at 11.59.48 AM.png

    Share this comment


    Link to comment
    Share on other sites

    @bobfa I think this has something to do with the perfect combination of sync type, file system, and filename. A lot of people have has issues with double quotes (") in the name over the years. now I know, after having the issue and looking it up. 

    Share this comment


    Link to comment
    Share on other sites

    9 minutes ago, The Computer Audiophile said:

    @bobfa I think this has something to do with the perfect combination of sync type, file system, and filename. A lot of people have has issues with double quotes (") in the name over the years. now I know, after having the issue and looking it up. 


    It might be more surprising if the algorithm “realized” it hadn’t backed up the files it failed to detect needed to be backed up.

    Share this comment


    Link to comment
    Share on other sites

    2 minutes ago, Jud said:


    It might be more surprising if the algorithm “realized” it hadn’t backed up the files it failed to detect needed to be backed up.

    HA! So true.

    Share this comment


    Link to comment
    Share on other sites

    OK, I re-ran the iDrive backup from scratch and have some information about number of files, size of backup and length of time it took to run.

     

    Backup Start Time: Sun Feb 12 22:04:05 2023

    Files considered for backup: 336482

    Files backed up now: 336482

    Backup Set Size: 15.04 TB

    Backup End Time: Wed Feb 15 09:36:24 2023

     

    That's roughly 15TB uploaded in 2.5 days if my math is correct.

    Share this comment


    Link to comment
    Share on other sites

    2 minutes ago, The Computer Audiophile said:

    OK, I re-ran the iDrive backup from scratch and have some information about number of files, size of backup and length of time it took to run.

     

    Backup Start Time: Sun Feb 12 22:04:05 2023

    Files considered for backup: 336482

    Files backed up now: 336482

    Backup Set Size: 15.04 TB

    Backup End Time: Wed Feb 15 09:36:24 2023

     

    That's roughly 15TB uploaded in 2.5 days if my math is correct.

     

    What's your max upload speed from your ISP if you know? Did you check during the upload to try to see actual speed?

     

    Here we've got 1.2Gbps download but only 40Mbps upload.

    Share this comment


    Link to comment
    Share on other sites

    6 minutes ago, Jud said:

     

    What's your max upload speed from your ISP if you know? Did you check during the upload to try to see actual speed?

     

    Here we've got 1.2Gbps download but only 40Mbps upload.

    I have 1Gbps up/down, without a cap, for $65 per month. 
     

    The upload was frequently around 80-108Mbps when I checked. The CPU of the QNAP was frequently around 60-90%. 

    Share this comment


    Link to comment
    Share on other sites

    Hey Chris, how often are you rebooting that NAS for firmware updates?

     

    I have the exact same version in a 6 bay, it failed after a reboot and would not power back on.  After doing some searching I discovered that my particular problem is quite a big issue for the x72xt NAS, they can potentially fail to power back on after a reboot.  Many people in other forums are saying to never upgrade the firmware or reboot them.

    Thankfully mine was still within warranty period.

     

    Point of my post I guess is to have a second backup, because if the NAS fails, you will ultimately need to get another QNAP NAS to recover files from, since my understanding is the way that data is stored is proprietary to the QNAP OS.

    Share this comment


    Link to comment
    Share on other sites

    On 2/26/2023 at 12:57 PM, robbbby said:

    Hey Chris, how often are you rebooting that NAS for firmware updates?

     

    I update the firmware whenever a new version is released, and reboot. I've had this NAS die on me once, and be replaced. I feel your pain :~)

    Share this comment


    Link to comment
    Share on other sites




    Create an account or sign in to comment

    You need to be a member in order to leave a comment

    Create an account

    Sign up for a new account in our community. It's easy!

    Register a new account

    Sign in

    Already have an account? Sign in here.

    Sign In Now



×
×
  • Create New...