Music library (backup) in the cloud?

[Qhwoeprktiyns]

I use google drive to back up my library (in addition to a backup on local drives).

I like the fact that file content is automatically indexed (either text files or images/pdf through OCR)

The automatic sync software on Windows works pretty well, but takes up a lot of resources on my computer when syncing. As mentioned by others, the initial upload of all the files is slow (takes a day for 1 terabytes). With automatic sync one has to be extra careful not to modify/delete content on Google Drive accidentally...

 

I did not find a good application to browse and play my music collection using Google Drive, so I developped my own.

 

I designed a web page to browse my music collection and play files either locally (using LMS) or through the cloud when I am not at home. I initially used the copy of the files on Google Drive, but soon found out that Google Drive is not the best platform for this, as access to the files is quite slow.

 

In addition to having a Google Drive backup, I store all album covers and album information in "buckets" on Google Cloud Storage, and selectively upload music files to Google Cloud Storage as well when I want to play an album "on the road". Here is an example: https://gentle-ridge-34867.herokuapp.com/album#1574857707026 (you cannot play the album - copyright issues... so I embedded the spotify album at the bottom). When run locally the web page allows you to edit the content of the page (which is simply a text file in markdown format stored in the album's folder). The content of the text file is parsed when saved and the credits are indexed and searchable (ex: find all albums were Jimmy Forrest played...).

Here is another example: https://gentle-ridge-34867.herokuapp.com/album#1567008961828

The composers are missing and plan on adding them.

 

[Qhwoeprktiyns]

I also wanted to avoid having an incoming internet connection to my home, and the need for a server running 24/7, so I developped this "hosted" solution... There are some solutions available to have a full cloud-based access to your library, which i looked into, but none of them were satisfactory in my opinion. Cloud computing is in its "infancy"...  

 

This is getting off topic, as it goes beyond the "backup" aspects, but maybe i will open another thread to discuss it. 

[Qhwoeprktiyns]

3 minutes ago, Dan Gravell said:

 

I'd be interested to learn more about what you are doing - I think "hybrid cloud" or "self hosting" in the cloud might be a growing phenomena.

 

Will open up a thread to describe what I am doing.

[Qhwoeprktiyns]

1 hour ago, bodiebill said:

 

@hopkins Did you ever start this thread about self-hosting?

 

Sorry, i had not taken the time to explain in detail.

 

Unfortunately what I am doing is a "custom" solution and I do not really see how it could be extended to others without investing significant time and effort into it. So i don't think a seperate thread is required. I'll explain here briefly.

 

I developped everything from scratch. My program scans automatically my folders. 

I store  album covers and associated artwork/pdf  on Google Cloud Storage. The "tags" and comments on my albums are stored in a PostgresQL database hosted on Heroku. 

 

The tracks themselves are for the time being only stored locally, but i plan on storing them as well on Google Cloud Storage, selectively for the time being, to allow me to  play them remotely.  For the time being, I play my files through LMS by sending the URLs of the files from my app. Works well, but it ain't Roon... 

 

That's about it

 

I put a few examples here: 

The example illustrate why i did all this. They are individual albums. I also have an "album grid" to browse my entire collection, but it is not finalized. I plan on adding artist bios/pictures as well - it all takes time. I spend more time "collecting" information on my ever-growing collection than on actually developping the solution. 

 

By the way, i am the author of this thread: https://community.roonlabs.com/t/bye-roon-why-i-am-not-renewing-my-subscription/67012

 

I followed through on my ideas and am very happy to have done so  I mostly listen to Jazz, and as a Jazz "collector", there are in my opinion many aspects which are just not handled by any music application today. Unfortunately,  there is no "collaborative" database that address these requirements. A shame, but that's the way it is.

 

 

[Qhwoeprktiyns]

Yes, and my solution is "hybrid' as some data is stored on the cloud and some locally. The app itself can be run either locally or from a remote server (on heroku) - the code itself is identical. 

[Qhwoeprktiyns]

11 minutes ago, The Computer Audiophile said:

I still can't believe HiFi companies haven't built cloud backup into their products yet. The Vox app (https://vox.rocks/) has had it for a long time. We pay premium prices for Roon, Aurender, Lumin, Auralic, etc... One would think the companies would just work with Backblaze or Amazon behind the scenes to white label the storage or just hook into the API and call it done. 

 

Agreed ! It really is not rocket science to develop a system that syncs your files to cloud storage. Having your associated metada (what Roon calls your "database") stored on the cloud as well is not an issue, given the bandwidth that most of us enjoy at home. For those who don't, it could be optional.

[Qhwoeprktiyns]

24 minutes ago, Jud said:

I am sorry to say the robustness of various backup solutions (multiple locations, remote governance and maintenance, etc.) will likely be tested in the coming weeks and months.

 

Cloud storage/apps arise happening everywhere else... Corporations are moving to cloud based solutions.

 

Having to run a home server 24/7 to listen to your music files (at home or remotely) seems "archaic" to me. A chromebook, a backup solution (local or remote) - that's all we should need.

[Qhwoeprktiyns]

18 minutes ago, AudioDoctor said:

I don't actually use mine to listen to music, just to backup. I could however.

 

That is indeed how many people do it today - and the subject of the thread.

In the coming years, we will probably see improved cloud based players, and the situation may be reversed, with "local" storage being purely for backup purposes, if at all.

[Qhwoeprktiyns]

33 minutes ago, bodiebill said:

 

Thanks @hopkins, sounds like an intricate custom setup!

 

It is  intricate in its "design", but it remains simple in its use: scan your folders,  browse, play, and edit your albums in a web page. 

 

For playback, I do plan on making it simpler with cloud based playback (have to check the cost of having all my collection on Google Cloud Storage or Amazon AWS), but as you may know audio playback in itself is a complicated topic... There are too many protocols (Web Audio, UPNP, LMS, casting, etc.. ), and some protocols/solutions are really poorly designed (Web Audio to start with...).

 

Back to cloud storage backup - sorry for the digressions...