Artificial intelligence-driven/automated tagging

[lamode]

Having recently spent several days on a tagging project (and I haven't even started with the genre tagging) I was thinking that this job could easily be automated to save a lot of time and to reduce errors.

 

I'm not talking about using an online tagging database because these online libraries contain problems of their own which still have to be corrected but at least they offer a starting point and even tags on commercial platforms such as Spotify or Qobuz are riddled with errors. A program could be written to scan a whole library and look for likely mistakes, e.g. artist tags which differ slightly but are likely to be the same could be flagged -> "I found separate tags for 'Beatles' and 'The Beatles'." Then you could select which tag to apply in all cases or choose to do nothing at all. Potential spelling errors could be flagged, as well as track numbers which accidentally found their way into track titles, etc. Other tags such as disk number and total disks are so time consuming to do manually but would be a snap with a smart tagging app.

 

Now for the really interesting part: there are already AI apps which can analyze a music track and identify the genre accurately (90-99% depending on the genre), not by looking up a database but by analyzing the actual music. THIS got my attention :) You can test this for yourself for free at https://tag.musiio.com/home  (unfortunately the demo only works with .mp3, .wav and .m4a file types – I have no idea whether this limitation applies to the paid plans). This service is not really aimed at the individual but surely intelligent tagging like this must be just around the corner for the music enthusiast and personally I can't wait!

[Qhwoeprktiyns]

I think there are already such applications, that can for example identify the "mood" of tracks, or similar attributes, and are used for automatic suggestions (radio). I don't personally find that of any interest, but others may.

 

Here's an example that popped up with a simple Google search - there must be many others: https://towardsdatascience.com/predicting-the-music-mood-of-a-song-with-deep-learning-c3ac2b45229e

 

There is also research being done on music analysis to identify the specificty of musicians (ex: what makes a Charlie Parker so identifiable and different from, say, a Johnny Hodges ?).   Not sure this is of general interest.

 

In terms of "tagging" (I prefer to talk about metadata), I spend a lot of time adding metadata to my albums - recording dates, venues, and credits. Those are not things that can be guessed, and they need to be accurate.

 

Provided there were an accurate repository for metadata, which does not exist today, and provided this repository were linked to the digital tracks (ex: AcurrateRip database), then one interesting application would be automatic recognition based on tracks, and not based on albums (as is the case for example with Roon). But that's a lot of ifs...

[Dan Gravell]

There are different _types_ of metadata. My own name for them are:

 

- Identification tags - identifies a release in some way (not necessarily uniquely, that could be a subset), e.g. ISRC, MB ID, a URI, album title, year, cat #

- Structural tags - defines the ordering and structuring of a release, e.g. disc number, track number, grouping title

- Classification tags - more subjective, e.g. genre, mood etc

 

In terms of identifying inconsistent data, this is already done by several apps.

 

The thing with auto suggesting tag values for classification tags, like detecting genre, is that while you might get "a" value you still need bounds within which values are acceptable. For example, within genre you don't want to end up with 200 genres - it just makes a music library unusable. These constraints could, of course, be automated.

[lamode]

2 hours ago, Dan Gravell said:

In terms of identifying inconsistent data, this is already done by several apps.

 

I would be interested to know which ones you would recommend for this purpose.

 

 

2 hours ago, Dan Gravell said:

The thing with auto suggesting tag values for classification tags, like detecting genre, is that while you might get "a" value you still need bounds within which values are acceptable. For example, within genre you don't want to end up with 200 genres - it just makes a music library unusable. These constraints could, of course, be automated.

 

That's a very good point. Some tags are insanely specific. The app which I linked to above seemed to identify several genres, if applicable, which would than allow you to automate the removal of overly specific tags, e.g. Jazz and Vocal Jazz could be later reduced to Jazz, if you so wished.

 

Still very messy though. Sigh...

[Qhwoeprktiyns]

6 hours ago, lamode said:

 

I would be interested to know which ones you would recommend for this purpose.

 

 

Bliss ? Dan's very nice app 😄

 

https://www.blisshq.com/

[lamode]

9 minutes ago, hopkins said:

 

Bliss ? Dan's very nice app 😄

 

https://www.blisshq.com/

 

Haha, I never look at signatures. Trying it now.

[WMW]

@lamode

 

Thoughts on Bliss?  I have an embarrassing large, sloppily-named and duplicate-saturated music collection I dream of having automagically fixed.  I'm looking for a reasonably accurate, stable, multiformat, multi-genre capable fixer for someone too impatient or stupid to master the other commercial organizers.  I suspect my problems relate to size, but dividing my collection from entire to artist in size hasn't helped.  

[AudioDoctor]

I have played around with Jaikoz and Songkong, haven't bought either though. They make a good starting point. I normally used it on a per album or per artist basis rather than on an entire library because there are always a few manual interventions needed and they are easier to find and fix when working on a smaller scale.

 

http://www.jthink.net/jaikoz/jsp/overview/startup.jsp

[lamode]

10 hours ago, WMW said:

@lamode

 

Thoughts on Bliss?

 

I tried it a month ago so unfortunately my memory of it isn't as fresh any more.

 

My impression was that I liked the overall concept and design of the app but it was let down by the quality of the data source upon which it relied. For example, one of my pet peeves is tags which combine multiple artists into one tag, e.g. a tag which creates a new artist tag of "Nancy Sinatra & Frank Sinatra" rather than creating two individual artist tags. Or, even worse, tags of the same pairing using a mixture of "and" and "&" for different albums.

 

In some cases, the database was just wrong. For example, this album cover indicates an artist name of "Buddy Bregman and his Orchestra":

 

 

 

However, according to Bliss, this artist name is incorrect and should be "The Buddy Bregman Orchestra"

 

 

 

Apart from the quality of the database, I liked it a lot and appreciated that it was possible to review suggestions before committing to them. For people with large collections which need extensive fixing, it could be a great starting point.

 

[Dan Gravell]

Thanks for this feedback @lamode!

 

The artists one is a common request. There are two ways of looking at it:

 

- Making sure incoming artist data splits into separate tags

- Analysing existing tags and split those

 

In general, this would be best implemented as a rule I think, because different players will have different demands - I've learnt how primitive some are, and support for multiple artist fields is not an option on some (my own car stereo only supports ID3v1 for example).

 

The artist name is an interesting one - the data is coming from https://www.discogs.com/Buddy-Bregman-And-His-Orchestra-Swinging-Kicks/master/450155 which lists the one from the cover as the canonical artist name for the release, but is also linked to https://www.discogs.com/artist/897724-The-Buddy-Bregman-Orchestra which is the canonical name for the artist - we use both as suggestions and that's why you have the two choices you showed in the screenshot. Maybe the latter should be a lower score, and so won't get the "recommended" tag. What do you think?

[lamode]

Hi @Dan Gravell, I wish now that I had posted feedback when my memory was fresh. The Buddy Bregman example was just the first that I found today and that is a particularly tricky example as back in those days the artist name could change a little with every release as they formed slightly difference groups and so there is an argument to be made for unifying the artist name (and this probably explains why Discogs links to the alternative name).

 

However, there are some better examples such as this mystifying suggestion to change the album artist to "Various Artists" :)

 

 

Even the alternative suggestion is wrong as it tries to correct "and" to "&" despite "and"  being used on the album itself.

 

 

 

The album cover for reference:

 

 

 

These are fairly small niggles and I would still rate Bliss above any other solution I've seen. I would recommend to users to take the time to check the recommendations.

[lamode]

Just a small example of Bliss being misled by the Discogs database. The track name is correct but Bliss would like to change the word "For" to "Fro".

 

 

This seems to be an error in the Discogs database.

 

 

To remove any doubt, here is the track listing on the CD itself:

 

 

I will try to submit a correction to Discogs for this example.

[lamode]

Something which is closer to an actual programme bug is the overloaded pop-up window which appears when hovering over a suggestion on a classical album. It becomes impossible to review the suggestion as the suggestion doesn't fit into the pop-up.

 

[Dan Gravell]

Yes, especially for classical releases!

Thanks for these items, they are nice and atomic so easy to add into our ticketing system.

[Jakob]

Hi guys - I just read through your very interesting discussion and I would like to share my two cents. 

 

Different types of metadata
Personally, I like this overview by SoundCharts. It differentiates metadata as following: 

1. Descriptive Metadata (song title, artist name, cover etc.)

2. Ownership/Performing Rights Metadata 

3. Recommendation Metadata (moods, energy, bpm etc.)

Automatic generated recommendation metadata
If you wish, you can try out the web app of Cyanite for free. We developed it in addition to our API services for everyone to try it out. You can just upload your MP3 and have it tagged by mood, genre, bpm, key, voice, instruments, musical era, energy level and emotional profile. The number of tags is constantly growing.

Shoot me a message if you have any question about further use cases, the technology behind - happy to help!

Greetings from Berlin

Jakob

Website: https://cyanite.ai/
Web App: https://app.cyanite.ai/library
Blog: https://cyanite.ai/blog-cyanite-ai/
Twitter: https://twitter.com/Cyanite_ai
LinkedIn: https://www.linkedin.com/company/cyanite-ai

 

[lamode]

On 6/15/2021 at 1:26 PM, Jakob said:

You can just upload your MP3 ...

 

 

Pity that the website only accepts MP3 for now. I tried to upload an album in MP3 format but the upload was rejected because the folder contained cover art and only MP3 files could be accepted. This means uploading selected tracks from each album, rather than uploading whole folders, which is a deal breaker for any significant number of albums.

[alandbush]

2 hours ago, lamode said:

 

Pity that the website only accepts MP3 for now. I tried to upload an album in MP3 format but the upload was rejected because the folder contained cover art and only MP3 files could be accepted.

All tracks in an album folder can be selected at the same time and then uploaded to the website, at least with Windows.

Nb - Microsoft Edge does not display the web page as intended.  Firefox does.

 

 

[WMW]

Re: MP3-only format.  I wondered about this.  I have no MP3 files.  One (ignorant of all things programming and most things in general) would think lossless formats would be amenable to such tagging, but perhaps with less speed owing to larger file size?

[Dan Gravell]

The only issue with speed would be upload/download. The actual tagging difference should be negligible.

 

Quote

Personally, I like this overview by SoundCharts

 

Plenty of good points here, but this is a different use case to music consumption which is the main concern here. In general I'd advise erring toward recording as little metadata as you need.

[Jakob]

13 hours ago, alandbush said:

All tracks in an album folder can be selected at the same time and then uploaded to the website, at least with Windows.

Nb - Microsoft Edge does not display the web page as intended.  Firefox does.

 

 

 

Exactly! Dropping the album folder doesn't work but selecting all Mp3s within the folder enables bulk uploading:).

 

16 hours ago, lamode said:

 

Pity that the website only accepts MP3 for now. I tried to upload an album in MP3 format but the upload was rejected because the folder contained cover art and only MP3 files could be accepted. This means uploading selected tracks from each album, rather than uploading whole folders, which is a deal breaker for any significant number of albums.

 

[Jakob]

9 hours ago, WMW said:

Re: MP3-only format.  I wondered about this.  I have no MP3 files.  One (ignorant of all things programming and most things in general) would think lossless formats would be amenable to such tagging, but perhaps with less speed owing to larger file size?

 

 Absolutely right. Losless files would result in much longer upload times due to their size. So mp3 are the best way to go here!

[katools]

@Jakob how is this intended to benefit a user - I'm guessing you license the tech to applications which in turn enables them to analyse a user's music and use your engine to create dynamic playlists based on a seed album/track/genre or artist/band?

[alandbush]

18 hours ago, katools said:

I'm guessing you license the tech to applications which in turn enables them to analyse a user's music and use your engine to create dynamic playlists based on a seed album/track/genre or artist/band?

 

Checkout the APM Music website for an example of how the AI is utilised.

[Jakob]

On 6/17/2021 at 9:31 PM, katools said:

@Jakob how is this intended to benefit a user - I'm guessing you license the tech to applications which in turn enables them to analyse a user's music and use your engine to create dynamic playlists based on a seed album/track/genre or artist/band?

 

Hi @katools - sorry for the long response time, your question slipped through somehow. You are totally right - in addition to our free web app, Cyanite is primarily designed to be used via our API. This allows music companies like streaming services, music production libraries or music software to use algorithms to analyze and recommend music without having to train and develop them themselves.

 

Here you can find a case study of how San Diego-based digital record pool BPM Supreme leverages the Cyanite API: https://cyanite.ai/2021/08/18/bpm-supreme-case-study/

 

I hope this helps and gives some more insights! Lemme know if more questions come up:) Cheers!

[lotusaurus]

In my ongoing efforts to find some tool to help me tag my music properly (by which I mean at least consistently), I came across MusiCHI, which seems orientated towards classical music and uses its own reference database for tagging.

https://musichieu.wordpress.com/about/

And there's a review here:

https://www.thewelltemperedcomputer.com/SW/Players/MusiCHI.htm

 

Unfortunately it seems to be dead - well, zombie-like. The developer has moved onto hardware, but in theory the software is still available. In the comments the developer says the software is still maintained and there is a demo version listed in the comments, but it doesn't work. I've been trying to get a demo by contacting the developer directly, who kindly provided a download link and customised login credentials, but the demo still won't run. I think the software can still be purchased. If anyone has had any recent experience with MusiCHI, would love to hear about it.

 

(Even though I couldn't get the software going, the developer's thoughts on the Zen of classical music tagging are very interesting - and amusing: https://musichieu.wordpress.com/tagging-audio-topics/)

 

But right now, bliss does look like a good option. For some reason I thought it wasn't a good fit when I first looked at it. Must take another look.