Local music collection app based on Discogs data

[Qhwoeprktiyns]

Frustrated with Roon, and seing the limits of all other software (based on tags), I embarked on writing an application that links my music collection to Discogs releases, and lets me browse through it using all the corresponding Discogs metadata.

 

It is not finished, but I would like to know whether this application could be of interest to others, in order to determine how much time and effort I should spend into it (to make it a "polished" product).

 

A few preliminary words on Discogs... It is far from perfect, but it does present a number of advantages:

- it contains more albums than any other database (both physical albums and albums offered through download only)

- the metadata "quality" is good - all data is referenced (artists, labels...), and it is widely used. Though the completeness of the data varies between various versions of an album, you can generally find a version with at least composition credits, musicians, and notes indicating the recording dates

- it manages some things fairly well which are not (yet) managed in Roon - labels on discs, series to identify box sets

- you can easily edit the data yourself - it is actually quite simple and "immediate"- you just have to follow some rules.

 

This is the way it will work...

 

First of all, it will require a sign-on on Discogs, and creating a "certificate" in order to interface with the application (takes a minute to do).

 

Discogs lets you manage your collection, and define "Libraries" in which you can store lists of releases (albums). Libraries can serve different purposes, but one of them will be used to identify albums within one's "physical" collection.

 

I do not intend on developping an automated solution to match one's music collection with Discogs releases. There are already applications available to do that (ex: SongKong). These can be used "offline" and the results can be uploaded into one's Discogs Libary (there is no upload function on the Discogs website, but I have developped one using the standard Discogs APIs). The association between a Discogs release and the local music files will be based on matching folders. I will be assuming that an album is contained within a dedicated folder on a local or network drive. The album folder, relative to a root folder, will be stored in a Discogs custom field.

 

So the process would be the following:

- to  initialize the data through an upload, use SongKong, or any other tool, to identify the Discogs releases corresponding to your album collection,  prepare a file with the album folder name, the Discogs release, and upload it into Discogs.

- Otherwise, add individual releases in your Library in Discogs, and then assign, within the application, to a local/network folder containing the album tracks 

The album folder name will be seperate from the name of the "drive" on which the folders are stored (it will be a "relative" path).

 

Once the folder is matched to a Discogs release, the appliation will assume that the order of the music files in the folder match the order of the tracks on the Discogs release. I plan on adding "alerts" if the number of tracks do not match, but do not plan on matching tracks more precisely (I do not see the need for it).

 

All the releases in one's library will be collected in the application, with all the corresponding metadata: titles, artists, contributors, etc...which can be used for browsing one's collection.

Custom fields defined in Discogs will also be available. I am already using one to store the original release date of albums, which I find useful. Custom fields can be used to identify the bitrare/sampling frequency in case one has several versions of the same album.

 

I do not intend on developping any mechanism to edit metadata within the application - that can all be done in Discogs. What needs to be customized can be managed through custom fields... None of the tags of the files will be used - purely Discogs data.

 

In terms of "playback", I am considering several options:

 

- playback through LMS (Logitech Media Server) - this will be possible as long as the local files can be read from the LMS server (through a shared network folder, for example).

- local playback (i.e. to the system sound cards), which is of limited use

- perhaps casting to UPNP endpoints (have not looked into it).

 

The application will be "standalone" (no client/server architecture), and will work on Mac OS, Windows, or Linux (I hope).

 

That's it for the moment...

 

Ultimately, I want to look into linking Discogs data with other external sources (Wikipedia, AllMusic...), using for example the Discogs "master" corresponding to a given "release", which is stored in WikiData along with links to other databases.  I am also working on building a database of discographies (especially for Jazz - that is my "pet" project), based on various internet sources, and have the ability to match discogs releases and tracks to that data, but that will be for later as well. One other possibility would also be to link artists and releases to streaming service data - but that is not a priority either.

 

Any input would be greatly appreciated. Does it make any sense ? Do you think it is useful ?...

 

Here is a simple screen shot of what it looks like now (work in progress)... The album detailed info will be added...

 

 

 

 

 

 

 

 

 

[One and a half]

I wish you well in the venture and would be interested in the Discogs reference and others database, compared to the Roon AllMusic (?) database source. Roon has this bad habit of changing album art to what it thinks is right, after I go to some trouble to track down the correct artwork (published country, time etc.)

 

Had your software arrived a few years ago BEFORE I embarked on a tedious job, but ultimately satisfying, metadata audit of some 30,000 tracks at the time, there would have been a really keen desire. Any new album that arrives, if dbpoweramp doesn't pickup the metadata, then Discogs database is consulted and the album ripped. Same for DSF files checked with MP3 Tag, also checked against Discogs database, but your software would check by itself, other than a manual start, which is OK for new albums.

 

I would recommend you have a chat with Jriver, their metadata application is worse than Roon's and about 10% of the time, 'get it wrong'. then again I don't expect a favourable answer, they can be quite abrasive.

 

The above works for pop, rock, jazz, genres, but classical has more demands which would be a challenge, of note long filenames for Windows 127 character limit.

 

I don't expect this work to be free, good luck and keep us posted.

[Qhwoeprktiyns]

I have come up with a better plan to match local files with one's Discogs Library.

 

Matching files with a personal Discogs library should in theory be simple and reliable if one were to use the number of tracks of an album, and track durations (given that one's personal Discogs library should be limited to the albums that are in your files). 

 

I have been trying to avoid desperately reading tag data from files  - simply because it adds a huge layer of complexity to my program. But if I use the data from whatever music server application I interface with, then I can easily get that data. I have already developped a link with LMS, to get all the album and track information, so I can set up an algorithm to match those with the information contained in the Discogs Library.

 

In Discogs, the track durations are approximate, because they are entered by "users", but between the number of tracks, and the duration, I could come up with a "best match".  I will test it out !

 

When the Discogs collection is updated, adding a release, removing one, then the potential candidates are even more limited and matching should be no issue (as long as one has included the "right" Discogs release). The algorithm will still be based on the assumption that a folder does not contain more than one release (but a release could be split over several folders).

 

This could also work with a UPNP server, or MPD server - just a question of time...

 

On a more technical note: one (very) annoying thing with Discogs, is that the API (interface) is limited to 60 calls per minute ! Obtaining the list of albums in a collection can be done with a few "calls" - one call returns a maximum of 100 entries - each entry containing the album title, artist, and a few other data. However, in order to get all the track information for each album, you need to use another "API", which only works album by album - so if you have 2000 albums in your library (approximately what I have), that means 2000/60 albums per minute, so roughly 33 minutes to scan the entire collection... That is not great - if you have 10.000 albums, that is roughly 3 hours... Getting the same information from LMS, for example, is a matter of minutes.

 

Obviously, one does not need to re-scan completely a collection every time, but it is still a "handicap".

 

Discogs does mention that more frequent "calls" can be granted on a case by case basis, so I guess there is some hope !

 

Concerning Jriver, I rather stick to "free" software for the moment, but will keep that in mind.

 

Concerning Roon, when you dig deep into the data, it is not always very reliable. The issue comes from AllMusic (and therefore Rovi, which is the service they interface with). I have albums on which the credits are wrong - they mix up musicians between different albums of the same artist. The point is, though, is that once misakes are found, you have no way of correcting them ! That is the good thing about Discogs, it is very easy to work with...

 

In terms of "presentation", Roon is really limited when it comes to box sets and generally multi-disc releases. That is true for classical music of course, but also for Jazz. I do not see how Roon is going to improve this point, but maybe someday they will...

 

Anyway, I like to have "control" of my music library, and I am just fed up with Roon, which in my opinion is very good for its integration with Tidal, but leaves a lot to be desired otherwise (and I happen not to have a Tidal account).

 

[Qhwoeprktiyns]

I did a quick check with my collection - in order to get a reliable match there are three criteria to consider:

- the number of tracks - no match possible if the number of tracks are different

- lowest average difference per track

- the difference in the duration of the complete album (between Discogs data and one's files)

 

The first two are probably sufficient. One potential issue is linked to the use of sub-tracks in Discogs - entry of track duration is not very consistent - but if an automated solution can get 95% of the album associated that should not be an issue.

 

 

 

 

 

 

[scorpio11]

Hi Hopkins, have you made any other changes to this project since you started? 

 

I would be interested to use a beta version if you have one available and I could make a comment for you

[Qhwoeprktiyns]

Hi, 

 

I abandoned the idea of using Discogs data some time ago. I build my app based on tags for a limited number of information and then add album metadata  on web pages. 

 

It is all described in my blog below, but it is my personal app and I do not intend on turning it into something others can use, as I am not a professional developer. I know some people working on similar (and better) systems, but nothing that is available today. 

 

 

 

[One and a half]

MP3TAG can reference the 'r' unique tag assigned to that album, and load the metadata including the cover art. The r number is located at the top right corner of the screen for that album and ties metadata for that release.

 

If there are multiple releases for an album, like Fleetwood Mac 'Rumours' the front page had an m number and MP3TAG will not use this, since multiple releases are catalogued at Discogs.

 

Tagscanner can also lookup discogs database , this is somewhat klunky, maybe I'm not setting it up correctly.

[Murl0c]

On 10/25/2021 at 11:17 PM, hopkins said:

 

I know some people working on similar (and better) systems, but nothing that is available today. 

Hey hopkins, hope you're fine!

Can you elaborate a bit more about who's working on these alternatives?

As you know, I am frustrated with Roon for the same reasons, so I'm still actively looking for a music player that could grab metadata from Discogs.

I would love to try these apps, and help these guys who're working on something similar.

 

Regards

 

murl0c

[Qhwoeprktiyns]

11 minutes ago, Murl0c said:

Hey hopkins, hope you're fine!

Can you elaborate a bit more about who's working on these alternatives?

As you know, I am frustrated with Roon for the same reasons, so I'm still actively looking for a music player that could grab metadata from Discogs.

I would love to try these apps, and help these guys who're working on something similar.

 

Regards

 

murl0c

 

I was not referring to apps, specifically linked to Discogs, unfortunately - sorry.