Creating a software tool to send audio analyzed data between different softwares

Pandamiac · October 1, 2021

Dear audiophile people out there,

this question is mainly for the digital audio enthusiasts. It will be a big challange for us to program this, so we apreciate every little hint that could help us to do this.

I am a multi media engineer with the main focus on audio engineering. Since i started to work with 3D applications (mainly Blender) i recognized the big potential to use audio signals to drive different parameters in my 3D software.

As far as i know there is no established expression for this, but i would call it "audio reactive motion graphics" and you can think about it like a really customizable audio visualizer. To get a better idea of what i mean, check some work iv´e done on my youtube channal where i load up music videos for my tunes (pandamiac official).

At the moment i use the tools, provided by Blender to analyze the sound. Unfortunately there are not that much more options to extract informations from the audio than the amplitude.

A friend of mine is an excellent programmer and we want to build a tool that enables me to send and recieve all kinds of mesurement data from DAWs to other programs (most important blender and later unreal engine 5) or extract it from rendered audiofiles. I could imagine that via a vst plugin on a chanal strip this could also work in realtime.

I am absolutly aware of that this is a big challange, but i am totaly ready to spend a lot of effort and research in this project.

So how to start?

My first thought was, that we dont have to invent the wheel again, if we could use some open source software to get the analyzing part done and then modify the code, so we could make it readable for the software, we want to send it to. The first challange is to find a good software for that. Do you have some recomendations?

If you have any experience with the stuff we´re trying to realize or you think my approach is totaly wrong pls let me know. Like i mentioned this will be only possible with the help of many different forum communitys and we really apreciate every contribution to this topic.

Greets :D

Pandamiac

Cebolla · October 1, 2021

Pretty niche, but not unheard of, eg the foobar2000 player has a well established set of visualisation plugins (projectM appears to be the latest & is open source based):

https://www.foobar2000.org/components/tag/visualization

How about making MQA signal detection a special case, with some devilish blue effects?