Article: Cloud Music Library Backup

[agladstone]

On 4/16/2021 at 12:02 PM, The Computer Audiophile said:

Also, the Backblaze B2 service is likely integrated into your NAS @jrobbins50. You can see pricing here for B2.

 

https://www.backblaze.com/b2/cloud-storage-pricing.html

I have to look into this! 
I’ve been using Google’s business level Google Workplace as Cloud backup in addition to a Synology NAS in Raid 5 with local sync and backup from my Aurender and also local external drives connected to a local desktop for my “core listening collection” and then also having that Synology NAS auto backup to and from my Google Workplace cloud. 
However, I can tell you from my extensive experience of downloading albums and collections from the cloud to my local “core collection”, that it takes a very long time to download from Google cloud and also there are a lot of errors often, so it requires double checking album by album and often re-downloading 2-4 tracks individually to fix these errors. 
I couldn’t begin to fathom what a Herculean task it would be if I ever actually needed to rely on a complete download of my Google cloud backup to my local storage and/or “core collection”, it would be a nightmare! 
Seems like it may be time for me to check out BackBlaze B2! 
I wonder if BackBlaze also has the ability to transfer / backup/ sync with Google (cloud to cloud)? 
I currently use MultCloud to sync and backup to and from Google Drive, MS Onedrive, Apple iCloud, and Dropbox automatically and it works great. 
My music collection is ENORMOUS, so my issues and situation is probably different than most others. 
The last I checked, I have about 560 TB’s of music in my collection and I have tried my best not to add much to that collection for at least the past 1-2 years since it’s become so expensive and time consuming to try and store and manage and tag all of it!! 
I only have about 80+ TB’s of this digital collection at home in my NAS (aka “core collection”) and it’s all backed up (but not on an automated or regular schedule) to a bunch of local external HDD’s. 
One of these days I really need to make an additional backup of it all and bring to a friend or relatives house for an offsite backup).  
Also, as others have mentioned, I have the top level subscriptions to Qobuz and Tidal, and as they’ve continued to get better and better and as my home internet has become Fiber and 1+Gbps, I’ve been listening to more and more HiRes streaming that sounds really good. 
Between these streaming services and the ability to download so many great and high quality recordings for your absolute favorite and best recorded albums from Qobuz, HDTracks, and many other online sources, I think if I suddenly lost my entire collection, I’d be able to “live with it”. 

[Johnseye]

One caveat to the Backblaze option I've found is that you can not backup a mapped drive (to your NAS) or from your NAS directly without the B2 package.  Going to B2 can increase the cost from $6/mo substantially depending on your storage and usage.  Of course a direct sync with the NAS has its own benefits

 

It does appear that Carbonite will allow backing up mapped drives from a Windows desktop.  So if you're looking to keep costs down that might be the best option.

 

Does this align with what others have found?

[Johnseye]

I take that back.  It looks like once you start getting into Safe Backup Pro level for Carbonite you don't get unlimited storage.

Carbonite Support Knowledge Base

 

So the Backblaze B2 solution is likely the most cost effective and feature rich.

[bluesman]

5 hours ago, Dan Gravell said:

If bit-rot is manifest as a change in the byte stream for any given file then yes, it could be transferred. This is precisely why snapshotting, as supported by the backup services (as opposed to most of the more generic and music focused services) is important. If you can find the last time the file(s) were good, you can roll back to that time. There may be many files affected.

As I understand it, bit rot is largely a problem in large scale, multilayer storage media. The risk of a discrepancy between what was written and what is read back (which is the functional definition of bit rot) is essentially zero from the HDDs and SSDs we use in consumer electronics.  The risk is greater (but still low) when the storage container is in the high TB to PB range, because so many layers have to be navigated to get to the target sectors.

 

If a read is corrupt, it will be corrupt when downloaded from cloud storage (where the capacity issue is very real).  The way to guard against this is twofold.  Keep a current local copy of your files, so you can replace corrupt archives, and keep a record of checksums so you can validate and verify the integrity of your files.  Storage systems with a file management layer can now check files automatically by comparing the checksum on upload to the checksum on accession.  I don’t know of any systems yet that will version and automatically retrieve the last known good version - but there probably are some and more will be coming as cloud storage capacity increases further.

 

How detailed you get is up to you.  Every file can be summed, or you can do it at the album level etc all the way up to a full disc or even an entire data transfer.  But the larger the bin, the harder it will be to find the affected bit(s).  So the practical alternative is to wait for files that throw errors and simply replace them ad hoc from your local backup.  That’s what I do, FWIW.

[Dan Gravell]

4 hours ago, agladstone said:

The last I checked, I have about 560 TB’s of music

 

Not sure whether to admire, laugh, cry... 😃

[AudioDoctor]

9 hours ago, Dan Gravell said:

This is a fair question. If bit-rot is manifest as a change in the byte stream for any given file then yes, it could be transferred. This is precisely why snapshotting, as supported by the backup services (as opposed to most of the more generic and music focused services) is important. If you can find the last time the file(s) were good, you can roll back to that time. There may be many files affected.

 

In addition, managing the integrity of your files, i.e. detecting bit-rot, is likely to be a separate management task dealt with by separate services.

 

I have versioned backups on my NAS, and those are backed up to an identical off-site NAS. Would bit-rot become evident when in the course of snapshotting, files that I have not recently changed start transferring to the NAS as changed files?

[AudioDoctor]

8 hours ago, agladstone said:

The last I checked, I have about 560 TB’s of music in my collection

 

Is that in a lossless compressed format such as FLAC, or uncompressed?

[bluesman]

21 minutes ago, AudioDoctor said:

I have versioned backups on my NAS, and those are backed up to an identical off-site NAS. Would bit-rot become evident when in the course of snapshotting, files that I have not recently changed start transferring to the NAS as changed files?

If your NAS units are all running simple storage, any corruption on the main NAS will be backed up off-site exactly as read.  If you're using automatic backup software that updates the off-site unit every time the main one changes, corruption will immediately trigger an update of the off-site unit to reflect the change (whether good or bad).  I suspect (but don't know for certain) that if your disks have proper error correction, bit corruption could trigger a remote update but autocorrection would do so again and restore the file to its correct form.  It probably depends on how fast your system is and what triggers the read that detects the error and fixes it.  I've never seen data on these processes, so I can't speculate on how reliable they are.

 

A simple RAID mirror will also see corrupt bits as a change and make the same change in the mirror disc.  As I recall, only RAID 6 will protect against "rebuilding" a good drive with corrupt data from the affected one.  But there are a few basics to keep in mind:

 

First, data decay is not very likely in the relatively small HDDS used for home NAS until they start closing in on their end of life.  Use SMART data to check on your drives, and replace them at the first sign of error correction.  I replace HDDs every 5 years even if they're working because the risk of data decay starts going up as they age.

 

Second, modern drives and the software that controls them use error correction code to identify flipped bits and remediate data decay.  Your SMART drive monitoring software will show you the rate of corrected errors - if that rate is too high (per the maunfacturer's instructions) or it's going up each time you check it over 3 consecutive times, replace the disk.  You can check your discs with utilities built into your OS or use 3rd party software like CrystalDiskInfo.  Scan your disks regularly and hope for this:

 

      

 

Third, SSDs have a different cause of data decay.  The insulating layer that keeps charged electrons where they belong degrades over time, and the bits flip.  So you need to follow the same replacement schedule for SSDs that you do for HDDs.  They both have finite data integrity periods, but for different reasons.

 

Fourth, heat accelerates this and most other memory, storage, and performance problems.  Run performance benchmarks every few months to be sure your computer isn't slowing down from undetected problems.  Make sure your computers, NAS units, etc are all in well ventilated spaces with good air flow around them.  Dust is a killer because it educes heat transfer to air - vacuum it all off of and out of everything.  Keep all fans clean and make sure they're functioning properly.  Monitor your drive temps just as you do (or should do) with your CPU and GPU temps. Use monitoring software that will alert you to potential problems.

 

[AudioDoctor]

I spent a good chunk of the day thinking about this and looking into all the set up. I have the "Master" music library here on my desktop on a large HDD. That gets synced to the music servers internal drive, and backed up to my local NAS, which then backs up to my NAS at my brothers house across town. They are versioned backups, so in theory I have multiple backups and should have good files...  I also checked drives on the 2011 Mac Mini that runs as a whole house Roon Server and while the drives passed (I think) it also looks like they need to be replaced. The thing still works though, so I may put a new SSD in it and plug a new external drive in as well. Or just use a new M1 mini, but that seems like massive overkill. I thought it would be a good idea to get checksums of the music files on the master as I am not sure that happens right now. I use Forklift to keep things in sync on my network and am still looking into whether or not it has that capability. Can anyone suggest a program that runs on an M1 mac that can generate checksums for files in a folder, with sub folders?

 

regarding checking the temps and general operational health of the various systems around my house, I am almost obsessive about that. Within a minute I can tell you the temps of any aspect of my computer, my CAPS, the mini, my M1 mini, and all the Raspberry Pi's in the house.

 

edit: When building the house I had what I consider to be a genius idea regarding the heat generated from the small room/closet that has the NAS and network gear in it. I ducted the heat from a duct in the ceiling, through a wall up above, and up into the heat exchanger. I can remotely check the temp in that room as well.

[agladstone]

On 4/19/2021 at 3:38 PM, AudioDoctor said:

 

Is that in a lossless compressed format such as FLAC, or uncompressed?

I have a lot of DSD, a lot of original vinyl rips in DSD and 24/96 FLAC, a ton of 24 bit digital downloads from Qobuz and HDTracks,etc. and the remainder is mostly 16/44 FLAC. 
Ive gotten to the point where I feel like I basically have almost everything ever recorded!! 
Lots and lots of rare live recordings and bootlegs too. 

[AudioDoctor]

43 minutes ago, agladstone said:

I have a lot of DSD, a lot of original vinyl rips in DSD and 24/96 FLAC, a ton of 24 bit digital downloads from Qobuz and HDTracks,etc. and the remainder is mostly 16/44 FLAC. 
Ive gotten to the point where I feel like I basically have almost everything ever recorded!! 
Lots and lots of rare live recordings and bootlegs too. 

 

Wow, I'm impressed.

[Mike27]

“...some services have been known to wipe tags from music files.”

 

Wait, what?

 

I’m well past the point I should have implemented some rigorous backup process. Swapping hard drives with my brother is one thing, but I’m better at accumulating things than organizing them. Metadata is just one part of the headache. I need revision control. I may have a half dozen versions of, say, an LP dub. Raw, clicks & noise repaired, an archival version and maybe one with a little EQ for general listening. Some random category that applies only to those files, perhaps. I may or may not want to keep all the intermediate versions. Keeping track via tags is difficult enough without something deleting them!

 

Yet I am not really surprised. Having just skimmed this article, I’ll dive in further soon. Thanks!

[Dan Gravell]

@Mike27 Yes, this is a thing. I should've linked to some examples actually because as it reads right now it seems like I've just attempted to insert a bit of FUD...

 

Exhibit A: YouTube Music https://www.reddit.com/r/musichoarder/comments/o1purr/recovering_metadata_from_youtube_music_library/

 

Exhibit B: Apple Music (when Match replaces with a lossless version; it appears possible to get the old version back) https://www.reddit.com/r/AppleMusic/comments/nwtyh4/apple_lossless_desktop_follow_up/

[agladstone]

16 hours ago, Mike27 said:

“...some services have been known to wipe tags from music files.”

 

Wait, what?

 

I’m well past the point I should have implemented some rigorous backup process. Swapping hard drives with my brother is one thing, but I’m better at accumulating things than organizing them. Metadata is just one part of the headache. I need revision control. I may have a half dozen versions of, say, an LP dub. Raw, clicks & noise repaired, an archival version and maybe one with a little EQ for general listening. Some random category that applies only to those files, perhaps. I may or may not want to keep all the intermediate versions. Keeping track via tags is difficult enough without something deleting them!

 

Yet I am not really surprised. Having just skimmed this article, I’ll dive in further soon. Thanks!

Google Drive leaves all metadata the same as what is uploaded (what I’m using for cloud backup). 
The only thing is that it does take “forever” to download backed up copies and it’s best to download one album at a time for FLAC/WAV/ ALAC, and I’ve discovered it’s best to download one track at a time for DSD or 24 bit FLAC/ALAC if it’s a large box set or similar. 

[Mike27]

@Dan Gravell: I was peripherally aware of the Apple “Match” hijinx, but not how convoluted. Hideous! Never considered YouTube because yuck. 
 

Bliss looks interesting, but I’m a notorious cheapskate. Historically, though, if I find something I really like, and can squeeze into the budget, who knows? Certainly some interesting info in that blog, thanks!

 

@agladstone: I have looked at Google Drive. Might be my best solution. They already know what I’m doing before I do. Or maybe Dropbox; I’ve used the free allotments for file transfer with no problem, except once someone invited me to share their space, only to find they put me on their employer’s corporate server, which then declared they owned my data before I even uploaded any... that didn’t last long. 
 

Meanwhile Windows networking somehow decided my Ethernet crossover cable is a public network, and blocked “rover” from “mothership” but not the other way round. Argh!!

 

Thanks for the replies!

[Dan Gravell]

I suspect the generic storage services will be much better in terms of not overwriting metadata - after all, they should just be storing data, not overwriting it as they see fit.

 

The music lockers, on the other hand, may have more convoluted approaches to managing your library, and this might be where issues creep in. I don't think they should but I find it unsurprising they do... if that makes sense.

[agladstone]

On 6/24/2021 at 5:52 AM, Dan Gravell said:

I suspect the generic storage services will be much better in terms of not overwriting metadata - after all, they should just be storing data, not overwriting it as they see fit.

 

The music lockers, on the other hand, may have more convoluted approaches to managing your library, and this might be where issues creep in. I don't think they should but I find it unsurprising they do... if that makes sense.

Dan:

Curious if you know anything or have any experience with SongKong ? 
I came across them in an article yesterday mentioning that it was being used with great success transferring music library from a server that stored all the meta data in a separate folder, and as a result when transferred to another brand music server it doesn’t bring over any of the metadata or album art. 
Apparently SongKong can scan the entire music library automatically after you transfer or download your music collection to a new server and it will re-tag and add album art automatically? 
Seems like this may actually be another solution to the situation mentioned here?  

[Dan Gravell]

On 6/25/2021 at 11:33 PM, agladstone said:

Curious if you know anything or have any experience with SongKong ? 

I came across them in an article yesterday mentioning that it was being used with great success transferring music library from a server that stored all the meta data in a separate folder, and as a result when transferred to another brand music server it doesn’t bring over any of the metadata or album art. 
Apparently SongKong can scan the entire music library automatically after you transfer or download your music collection to a new server and it will re-tag and add album art automatically? 
Seems like this may actually be another solution to the situation mentioned here?  

 

Yep - it is - and I know the author!

[agladstone]

3 hours ago, Dan Gravell said:

 

Yep - it is - and I know the author!

Thanks! Maybe I’ll give it a try! I have about 80 TB’s of downloads on a series of external drives that I suspect are full of duplicates that I’ve been too lazy to attempt to look through for 1+ years now, should be the perfect place to experiment with it! (It apparently can manage duplicates automatically in addition to auto-tagging meta data and auto adding album art).