Jump to content
IGNORED

Article: Cloud Music Library Backup


Recommended Posts

Thanks, @Dan Gravell.  As Chris knows, I have a quite large (over 35 Tb) music collection on my Synology NAS which has fault tolerance for two hard drives to fail simultaneously, but no offsite storage. I’m sure offloading a copy of my library over the cloud would take days if not weeks, but that’s not my real concern. I’m guessing the cost to maintain a cloud-based backup of a library this size must be out of control. How would you approach such a beast?  Thanks. JCR 

Link to post
Share on other sites

Nice article!

 

I find using the "sync storage services", without actually "synching" to be a good option for those who have relatively small music libraries (full upload < 1 day) with files that are not constantly being changed (i.e. tags modified). The obvious drawback is that you need to remember to manually add your new files/changed files regularly, the advantage is that you don't risk accidentally replicating file deletions in both your local storage and cloud storage (as you pointed out). For "smallish" libraries, you could regularly do a full backup. For large libraries, especially when files are frequently updated, this options is probably not workable. You can also deactivate "synching" and reactivate it when needed.

 

These types of issues, by the way, highlight the fundamental disadvantage of having music metadata managed within music files (as tags) - but that's an entirely different topic :)

 

 

my blog

 

Link to post
Share on other sites
20 minutes ago, hopkins said:

I find using the "sync storage services", without actually "synching" to be a good option for those who have relatively small music libraries (full upload < 1 day) with files that are not constantly being changed (i.e. tags modified). The obvious drawback is that you need to remember to manually add your new files/changed files regularly, the advantage is that you don't risk accidentally replicating file deletions in both your local storage and cloud storage (as you pointed out). For "smallish" libraries, you could regularly do a full backup. For large libraries, especially when files are frequently updated, this options is probably not workable. You can also deactivate "synching" and reactivate it when needed.

 

You have to be careful with that though. If you don't automate this there's a good chance you'll forget, or something else will get in the way. Humans are best at creativity, not routine - we get easily bored - "I'll just skip it this month, it'll be fine". I know, I know, some of us are better than others but...

 

Put another way, it depends on the value of the library.

 

bliss - fully automated music organizer. Read the music library management blog.

Link to post
Share on other sites
Just now, Dan Gravell said:

 

You have to be careful with that though. If you don't automate this there's a good chance you'll forget, or something else will get in the way. Humans are best at creativity, not routine - we get easily bored - "I'll just skip it this month, it'll be fine". I know, I know, some of us are better than others but...

 

Put another way, it depends on the value of the library.

 

Absolutely - and I just started an other full upload after reading your article !

my blog

 

Link to post
Share on other sites

Excellent article, @Dan Gravell, something which guides the reader fluently through a decision path/process.
 

I have been there lately and decided against online backup at the moment, but I may look for a selection of albums and playlists to be backed up in the cloud in the future.

I am a far shot away from the collection size of @The Computer Audiophile or @jrobbins50, however including all other data incl. photos and videos as a driver demand is surely above 10TB, but the rate of data growth has been slowed down in the last 3 years, mostly by fiber connection and corresponding streaming services for video and music, while having arrived concomitant at a fairly complete collection for my needs at the other hand. growth drivers have been rather photos & mobilke phone backups  lately ...


After re-visiting some of the helpful threads in the forums, I chose to upgrade my live NAS and to redress my 2012 desktop machine into a NAS using any disk above 1TB I had acquired since 2009 as backup. Plan is to do an automated backup once a week. Number two will be a mini-ITX-tower using 3 oversized HDDs. This one will be stored at a different location and only synced every other month.
 

As I feel that cloud data services (perhaps except for the Norwegian ones) have proved unreliable in terms of price, quantity, customer data protection and life cycle, I may consider using them only for selected items which are the core of the music I am usually listening to. Which is far  less than 5 % of the collection and would be more playlist oriented than complete albums.

During the process of analyzing and planning I have learned how much data our family's digital life has assembled in the last 25 years. There are obviously some parts more than doubled by older incoherent backup strategies, however losing them incidentally would be something great of a drama and imaging that scenario also convinced the treasury that an investment into back up tech is a wise transfer. Far easier to negotiate than buying a new pair of loudspeakers. ... cheaper too ...

One important driver for taking care of the data available in our household and in the cloud, has been the example of Tidal (suddenly but not unexpected) changing the format of the streaming files into lossy and MQA's attempt into dominating the consumers' choices. Being able to rely on the lossless format of my music in the future and to pass it on to my sons, imho,  is the most serious driver for me and demands a coherent back up planning.

Cheers, Tom

 

Cheers, Tom

Link to post
Share on other sites
1 minute ago, DuckToller said:

During the process of analyzing and planning I have learned how much data our family's digital life has assembled in the last 25 years. There are obviously some parts more than doubled by older incoherent backup strategies, however losing them incidentally would be something great of a drama and imaging that scenario also convinced the treasury that an investment into back up tech is a wise transfer. Far easier to negotiate than buying a new pair of loudspeakers. ... cheaper too ...

 

 

Very much so - when I think of all our family photos and videos... and, of course, the "sync" and "backup" services can be used for non-audio media.

 

bliss - fully automated music organizer. Read the music library management blog.

Link to post
Share on other sites

I find this to generally be a good article. Perhaps my question is too technical and beyond the scope of this article, but... How do we address bit rot? Is that something that can be transferred between devices in such a way that the backup file would also be harmed somehow? I have been thinking about this as drives I have are aging to beyond a decade.

No electron left behind...

Link to post
Share on other sites

Great article, thanks @Dan Gravell

I learned a few things re acronyms too 😁!

e.g.: 

  • PEBKAC ("problem exists between keyboard and chair")
  • identity error or ID-10T/1D-10T error ("idiot error"), 
  • PICNIC ("problem in chair, not in computer"), 
  • IBM error ("idiot behind machine error") and other similar phrases are also used as slang in technical circles with derogatory meaning

 

I am currently rebuilding a 1TB drive at a significant $ and time cost,  that was subject to my IBM error, so very timely article.

 

Link to post
Share on other sites
11 hours ago, The Computer Audiophile said:

Also, the Backblaze B2 service is likely integrated into your NAS @jrobbins50. You can see pricing here for B2.

 

https://www.backblaze.com/b2/cloud-storage-pricing.html

 

I started with Backblaze Personal but switched to Backblaze B2 recently because it's so easy to automate from my Synology Diskstation.  Very fast uploads, reasonable prices.

John Walker - IT Executive

Headphone - MacMini running Roon Server > Netgear Orbi wireless > Blue Jeans Cable Ethernet > mRendu Roon endpoint > iFi Audio xDSD + iFi Audio xCAN > Focal Elegia

Home Theater - Mac Mini running Roon Server > Blue Jeans Cable HDMI > Pioneer Elite SC-81 > MartinLogan Motion series home theater speakers + M&K subwoofer

Link to post
Share on other sites

Thanks for the writeup. It looks like prices have come down and stabilized with multiple offerings. My preference is to backup to a platform like Backblaze or Carbonite, but I haven't done this yet. Does anyone know if it's possible to map a drive to these services or connect a player like Roon to that cloud storage? It would be great if I could use it as a source and not just a backup space. 

Link to post
Share on other sites
1 hour ago, Johnseye said:

Does anyone know if it's possible to map a drive to these services or connect a player like Roon to that cloud storage? It would be great if I could use it as a source and not just a backup space. 

I’ve done this but the speed and latency sucks. It isn’t a design that equivalent to a streaming service. 

Founder of Audiophile Style

Announcing The Audiophile Style Podcast

Link to post
Share on other sites
7 minutes ago, The Computer Audiophile said:

I’ve done this but the speed and latency sucks. It isn’t a design that equivalent to a streaming service. 

 

Even with buffering is it unlistenable? Or are you better off with a tunnel and using something like JRiver to connect to your home NAS?

 

Link to post
Share on other sites

OraStream used to offer a service for this purpose. I haven't kept up with them in recent years but they may still have it. They would also stream your library to you as well.

Link to post
Share on other sites
39 minutes ago, bobflood said:

OraStream used to offer a service for this purpose. I haven't kept up with them in recent years but they may still have it. They would also stream your library to you as well.

 

Thanks, that would be along the lines of what I'm looking for. If only their cloud storage was more than 1 TB.

Link to post
Share on other sites
On 4/16/2021 at 10:13 PM, AudioDoctor said:

How do we address bit rot? Is that something that can be transferred between devices in such a way that the backup file would also be harmed somehow? I have been thinking about this as drives I have are aging to beyond a decade.


This is a fair question. If bit-rot is manifest as a change in the byte stream for any given file then yes, it could be transferred. This is precisely why snapshotting, as supported by the backup services (as opposed to most of the more generic and music focused services) is important. If you can find the last time the file(s) were good, you can roll back to that time. There may be many files affected.

 

In addition, managing the integrity of your files, i.e. detecting bit-rot, is likely to be a separate management task dealt with by separate services.

 

bliss - fully automated music organizer. Read the music library management blog.

Link to post
Share on other sites
On 4/18/2021 at 1:26 AM, bobflood said:

OraStream used to offer a service for this purpose. I haven't kept up with them in recent years but they may still have it. They would also stream your library to you as well.

 

If you mean BRIO, that would be a type of music locker - it's a useful redundant copy, but as I have said it is not a full backup service. Depending on the value of your collection it may or may not be appropriate.

 

bliss - fully automated music organizer. Read the music library management blog.

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now


×
×
  • Create New...