John Dyson Posted March 26, 2020 Author Share Posted March 26, 2020 More 'good' news -- I have two methods available for built-in playing while on Windows. Linux is already all figured out, of course -- I use it!!! TWO kinds of 'play' facility are included because --play and --splay are very useful features. I am not confident about my Windows expertise and have little recent Windows experience, so I am providing QUANTITY that is probably replacing the more desirable QUALITY :-). I wouldn't have the patience to use the decoder without realtime play, and that is probably the major reason why the low-quality mode in the decoder is supported... Maybe the slowest i5 or maybe even i3 with two full speed cores could probably run in realtime in the --basic mode. Running in real-time means that the decoder can play results immediately as it is decoding... This allows quick tweaking/changing modes and filters to get the 'best' results. The two real-time play mechanisms -- I am including a SOX download (with source, pointers, attribution)), and also my own basic, silly little utility. The version of SOX being used is the WINDOWS native binary, so there is no need for more cygwin cruft -- if more than the very basic 'cygwin' is needed, then we get into heavy licensing issues. SO, NEITHER of the play facilities depend on more than minimal 'cygwin' support. The 'easy to use' SOX IS indeed a cygwin verison, and it took quite a bit of hackery to figure out why the 'standard' Windows SOX doesn't properly realtime play. The cygwin version will NOT be used even it is installed on a system, the totally standard latest Windows release is the only version of SOX that will work with the decoder. * For those interested in independently using the Windows SOX binary to play music files -- I divined the secrets, and also even got it working on pipelines. There are a few glitches in the native Windows version of SOX... I'll provide that information when this all settles down. Both player versions THEORETICALLY support selecting non-default audio devices for output, but it appears that the SOX version is the only one that works for me. Both versions appear to work on the default device. Basically, to use the 'SOX' version, a totally standard *Windows* SOX installation needs to be done -- I am including the files and sources to be downloaded separately, and the SOX must be in the standard location. My little utility is included in the decoder distribution along with source code and library source code (per GPL.) The utility is stand alone, but doesn't work very well by itself. The SOX version might be of some non-DA use. If you are security conscious -- just download the 14.4.2 version of SOX and install it from the main SOX site. Mine is identical. Invoking a playout to the standard device using SOX, just add the '--splay' switch. Invoking a playout to the standard device using the silly little adjunct program, just add the '--play' switch. More to come -- I have all the ducks in order for the 'play' stuff, so the next step is the Linux version. That is only packaging, and with a little luck will be ready tonight. John Link to comment
John Dyson Posted March 27, 2020 Author Share Posted March 27, 2020 I just uploaded new version of FA decoder, also Linux version and SOX installation. I'll send more complete information tomorrow morning, but letting you all know ASAP. Quick instructions: The windows version works just like previous one, except maybe the --play option will work. It also supports using sox to directly play the output by using the --splay switch instead. For --splay to work, must install the standard SOX as included in the repository. You can instead install the exact same version from the SOX site -- I made ZERO changes, exact same files. --splay takes an argument like --splay=1 or somesuch to select the output device. The Linux version simply unpacks into a subidrectory using tar... SImply: >> tar xovf filename.tar.bz2 will produce the subdir containing the decoder... It is EXACTLY the version, same binary that I use for testing. The newcommands files has updated decoding parameters for numerous sources... Basically, --fa=2 is a good first choice for the decoding mode. Here is the same directory as before.... This is probably very confusing unless you have read the previous instructions, then it is probably just confusing... The FA decoding is slightly improved, the command lines are a little more simple. The realtime play has a better chance of working. Good luck (you'll need it until I update this information more in the morning!!!) :-). https://www.dropbox.com/sh/1srzzih0qoi1k4l/AAAMNIQ47AzBe1TubxJutJADa?dl=0 John lucretius 1 Link to comment
lucretius Posted March 27, 2020 Share Posted March 27, 2020 4 hours ago, John Dyson said: I just uploaded new version of FA decoder, also Linux version and SOX installation. I'll send more complete information tomorrow morning, but letting you all know ASAP. Quick instructions: The windows version works just like previous one, except maybe the --play option will work. It also supports using sox to directly play the output by using the --splay switch instead. For --splay to work, must install the standard SOX as included in the repository. You can instead install the exact same version from the SOX site -- I made ZERO changes, exact same files. --splay takes an argument like --splay=1 or somesuch to select the output device. The Linux version simply unpacks into a subidrectory using tar... SImply: >> tar xovf filename.tar.bz2 will produce the subdir containing the decoder... It is EXACTLY the version, same binary that I use for testing. The newcommands files has updated decoding parameters for numerous sources... Basically, --fa=2 is a good first choice for the decoding mode. Here is the same directory as before.... This is probably very confusing unless you have read the previous instructions, then it is probably just confusing... The FA decoding is slightly improved, the command lines are a little more simple. The realtime play has a better chance of working. Good luck (you'll need it until I update this information more in the morning!!!) :-). https://www.dropbox.com/sh/1srzzih0qoi1k4l/AAAMNIQ47AzBe1TubxJutJADa?dl=0 John Thank you. Both --play and --splay (with no arguments) are working fine on my machine (Windows 10). I'll get around to installing the linux version later. I cannot wait to hear what those glitches are in the native Windows version of SOX! All the best! mQa is dead! Link to comment
John Dyson Posted March 27, 2020 Author Share Posted March 27, 2020 7 hours ago, lucretius said: Thank you. Both --play and --splay (with no arguments) are working fine on my machine (Windows 10). I'll get around to installing the linux version later. I cannot wait to hear what those glitches are in the native Windows version of SOX! All the best! Oh -- the glitches -- sorry I didn't provide more complete info last night -- I was barely awake :-). It will stil be a few more hours before I write more about using the decoder, but it looks like you are doing great so far.. I am SO glad that both version of play worked, my Windows expertise has rotted away over the last 20yrs :-). Okay, about sox... The 'trick' is that SOX doesn't know which driver to use. At first, as you know, it seems like Windows SOX almost doesn't even support a 'play' feature. Note that I am not directly using a play command when I execute the sox command from the decoder, but I use the exact raw sox command as follows (note the paranoia in setting the I/O mode, might not be needed); * actually, instead of 'sox', I am using the full path, so the sox must be installed in the normal place for the version as follows from the source code: static const char *soxlocation = "c:\\Program Files (x86)\\sox-14-4-2\\sox.exe"; or for humans: c:\Program Files (x86)\sox-14-4-2\sox.exe pipe input | sox -q --type=wav - --bits=16 -twaveaudio rate 44100 or pipe input | sox -q --type=wav - --bits=16 -twaveaudio <id> rate 44100 (<id> is an index like 0,1,2,3,4 which points to the desired output device) A more normal use of sox, just to play a file, one would do: sox filename.wav -twaveaudio the above should work, and shouldn't really need to set the bits modes or the data rate. I set the data rate above because the decoder writes at 88.2k/floating point, and I wasn't sure if all versions of windows drivers supported that mode. Another mod that might be helpful, instead of '-twaveaudio', you might need '-t waveaudio' (note the space.) Another oddity -- did you notice that I specified the INPUT data type from the pipeline? Normally, on Unix, one doesn't need to specify the input stream type as autodection works okay on input streams. However, on Windows, the automatic type detection seems to fail. SO, I forced the input type to be '--type=wav'. The type detection seems to work okay when using a normal file instead of '-' or pipe for the input file. About how I run 'sox' indirectly from the DHNRDS DA... Basically, I stay in the POSIX land instead of using direct Windows calls. I cannot use the 'system(3)' call because 'system(3)' calls the shell, but most people aren't going to have a POSIX/cygwin 'bash' program on their systems, so I get deeper into the POSIX nitty/gritty and do the old fork(2)/execl(2) thing and manually set up the call to 'sox'. The aoplay.exe program is called in a similar way. The data stream when using SOX is a standard .wav file, but on aoplay, since I didn't want to encumber my .wav file processing subrotines (however crappy my subroutines are), I simply created a stream that is led with a 'magic number', then a data rate and then an extra data item for the future. The rest of the stream are the audio bytes encoded as 24 bit signed data... Basically, the aoplay.exe uses 24bit signed .wav data, but without a normal .wav header. More info coming as I 'get my engine started' later this morning or afternoon. John lucretius 1 Link to comment
lucretius Posted March 28, 2020 Share Posted March 28, 2020 12 hours ago, John Dyson said: Oh -- the glitches -- sorry I didn't provide more complete info last night -- I was barely awake :-). It will stil be a few more hours before I write more about using the decoder, but it looks like you are doing great so far.. I am SO glad that both version of play worked, my Windows expertise has rotted away over the last 20yrs :-). Okay, about sox... The 'trick' is that SOX doesn't know which driver to use. At first, as you know, it seems like Windows SOX almost doesn't even support a 'play' feature. Note that I am not directly using a play command when I execute the sox command from the decoder, but I use the exact raw sox command as follows (note the paranoia in setting the I/O mode, might not be needed); * actually, instead of 'sox', I am using the full path, so the sox must be installed in the normal place for the version as follows from the source code: static const char *soxlocation = "c:\\Program Files (x86)\\sox-14-4-2\\sox.exe"; or for humans: c:\Program Files (x86)\sox-14-4-2\sox.exe pipe input | sox -q --type=wav - --bits=16 -twaveaudio rate 44100 or pipe input | sox -q --type=wav - --bits=16 -twaveaudio <id> rate 44100 (<id> is an index like 0,1,2,3,4 which points to the desired output device) A more normal use of sox, just to play a file, one would do: sox filename.wav -twaveaudio the above should work, and shouldn't really need to set the bits modes or the data rate. I set the data rate above because the decoder writes at 88.2k/floating point, and I wasn't sure if all versions of windows drivers supported that mode. Another mod that might be helpful, instead of '-twaveaudio', you might need '-t waveaudio' (note the space.) Another oddity -- did you notice that I specified the INPUT data type from the pipeline? Normally, on Unix, one doesn't need to specify the input stream type as autodection works okay on input streams. However, on Windows, the automatic type detection seems to fail. SO, I forced the input type to be '--type=wav'. The type detection seems to work okay when using a normal file instead of '-' or pipe for the input file. About how I run 'sox' indirectly from the DHNRDS DA... Basically, I stay in the POSIX land instead of using direct Windows calls. I cannot use the 'system(3)' call because 'system(3)' calls the shell, but most people aren't going to have a POSIX/cygwin 'bash' program on their systems, so I get deeper into the POSIX nitty/gritty and do the old fork(2)/execl(2) thing and manually set up the call to 'sox'. The aoplay.exe program is called in a similar way. The data stream when using SOX is a standard .wav file, but on aoplay, since I didn't want to encumber my .wav file processing subrotines (however crappy my subroutines are), I simply created a stream that is led with a 'magic number', then a data rate and then an extra data item for the future. The rest of the stream are the audio bytes encoded as 24 bit signed data... Basically, the aoplay.exe uses 24bit signed .wav data, but without a normal .wav header. More info coming as I 'get my engine started' later this morning or afternoon. John Thanks. As long as I use '-twaveaudio', SoX play will work (whether invoked as 'sox' or 'play'). There seems to be no advantage in invoking sox as play, since all the same syntax is required (and specifying '-t' and 'waveaudio' is necessary). mQa is dead! Link to comment
lucretius Posted March 28, 2020 Share Posted March 28, 2020 @John Dyson Now I am trying the linux version. I have Ubuntu 18.04 LTS installed in a VM. I already have SoX installed. Forgive my stupid question but in which linux directory should I place 'da-avx'? Thank you! mQa is dead! Link to comment
lucretius Posted March 28, 2020 Share Posted March 28, 2020 1 hour ago, lucretius said: @John Dyson Now I am trying the linux version. I have Ubuntu 18.04 LTS installed in a VM. I already have SoX installed. Forgive my stupid question but in which linux directory should I place 'da-avx'? Thank you! Actually, I'm having a problem making the 'da-avx' executable in Ubuntu. (I don't use Ubuntu much, LOL.) mQa is dead! Link to comment
John Dyson Posted March 28, 2020 Author Share Posted March 28, 2020 1 hour ago, lucretius said: @John Dyson Now I am trying the linux version. I have Ubuntu 18.04 LTS installed in a VM. I already have SoX installed. Forgive my stupid question but in which linux directory should I place 'da-avx'? Thank you! da-avx should be able to sit anywhere, but I didn't supply the aoplay because I usually just pipe the output to the normal Linux sox play command. You MIGHT have shared library issues -- I am not sure, it depends on the version of Linux, how recent/etc. My version of Linux is fairly old (a few years) so should be compatible with your shared libs. Worst comes to worse, we might be able to put together a version that you can link, but I hope we don't have to go that far!!! John Link to comment
John Dyson Posted March 28, 2020 Author Share Posted March 28, 2020 2 minutes ago, lucretius said: Actually, I'm having a problem making the 'da-avx' executable in Ubuntu. (I don't use Ubuntu much, LOL.) Ohhhh.... you might have to 'chmod +x da-avx'... Also, you REALLY need the Intel Haswell or newer CPU (i4000 or newer), Zen or Excavator. AVX2 is importamt, even though I called the binary da-avx. I can make an sse3 version if needed, I don't have an sse3 machine to test on for Linux, but I can use the same settings as I use for Windows SSE3 (stuff like Atoms.) BTW, I also don't have an AVX2 machine for Windows, but Richard does... SO, I have done so much testing through Richard that I know what I can get by with. However, I can make an SSE3 version based upon the Windows version -- no problem, if it will help. Just tell me the kind of CPU that you are using... Happily, I am using a REALLY REALLY good vectorizing compiler (Clang++) so the code generated is the very best available. g++ is much slower for some reason, even though it is a little easier to use. (Sorry for the lack of CPU dispatch, but the code is so very tight because the algorithms are incredibly itntense that CPU dispatch would really slow down the code.) John Link to comment
lucretius Posted March 28, 2020 Share Posted March 28, 2020 43 minutes ago, John Dyson said: Ohhhh.... you might have to 'chmod +x da-avx'... Also, you REALLY need the Intel Haswell or newer CPU (i4000 or newer), Zen or Excavator. AVX2 is importamt, even though I called the binary da-avx. I can make an sse3 version if needed, I don't have an sse3 machine to test on for Linux, but I can use the same settings as I use for Windows SSE3 (stuff like Atoms.) BTW, I also don't have an AVX2 machine for Windows, but Richard does... SO, I have done so much testing through Richard that I know what I can get by with. However, I can make an SSE3 version based upon the Windows version -- no problem, if it will help. Just tell me the kind of CPU that you are using... Happily, I am using a REALLY REALLY good vectorizing compiler (Clang++) so the code generated is the very best available. g++ is much slower for some reason, even though it is a little easier to use. (Sorry for the lack of CPU dispatch, but the code is so very tight because the algorithms are incredibly itntense that CPU dispatch would really slow down the code.) John I tried 'chmod +x da-avx'. However, when I try to run 'da-avx', I am still getting "command not found". [Also, my CPU has AVX2 although I am using a VMware Workstation VM.] Maybe it's something to do with Ubuntu? OTH, Sox 'play' works fine in Ubuntu. I can just run 'play filename.wav' -- without 'waveaudio' option, the latter doesn't work -- waveaudio option must be unique to Windows? mQa is dead! Link to comment
John Dyson Posted March 28, 2020 Author Share Posted March 28, 2020 14 minutes ago, lucretius said: I tried 'chmod +x da-avx'. However, when I try to run 'da-avx', I am still getting "command not found". [Also, my CPU has AVX2 although I am using a VMware Workstation VM.] Maybe it's something to do with Ubuntu? OTH, Sox 'play' works fine in Ubuntu. I can just run 'play filename.wav' -- without 'waveaudio' option, the latter doesn't work -- waveaudio option must be unique to Windows? About running da-avx. Ahhh... the answer: try ./da-avx... Might have to specify the full path when in the current directory. In normal 'Unix' installations, it has become recent (since early 1990's) tradition to keep the current directory out of the path. That is for the early attempts at improving security. I usually reconfigure my accounts where I do development to have the OLD style of path. I do NOT use that for root or accounts that have priv access though. About the 'waveaudio' strangeness... Yes, that is just a windows thing. On Linux, it appears that sox properly chases down the list of available drivers and choses the correct one. On the normal 'Windows' build of SOX, it seems to be unable to chase down the correct driver. The problem seems simialr WRT being able to sense the file type of a pipe input... The cygwin version of SOX works like the Unix/Linux version, but then the cygwin version needs so many .dlls as to make it a licensing nightmare. (I was considering making the cygwin version available, but then the user would have to install that huge morass -- don't want to do that.) Let me know if ./da-avx works.... Otherwise, I'll build the SSE3 verison -- if that is the problem. John Link to comment
lucretius Posted March 28, 2020 Share Posted March 28, 2020 49 minutes ago, John Dyson said: About running da-avx. Ahhh... the answer: try ./da-avx... Might have to specify the full path when in the current directory. In normal 'Unix' installations, it has become recent (since early 1990's) tradition to keep the current directory out of the path. That is for the early attempts at improving security. I usually reconfigure my accounts where I do development to have the OLD style of path. I do NOT use that for root or accounts that have priv access though. About the 'waveaudio' strangeness... Yes, that is just a windows thing. On Linux, it appears that sox properly chases down the list of available drivers and choses the correct one. On the normal 'Windows' build of SOX, it seems to be unable to chase down the correct driver. The problem seems simialr WRT being able to sense the file type of a pipe input... The cygwin version of SOX works like the Unix/Linux version, but then the cygwin version needs so many .dlls as to make it a licensing nightmare. (I was considering making the cygwin version available, but then the user would have to install that huge morass -- don't want to do that.) Let me know if ./da-avx works.... Otherwise, I'll build the SSE3 verison -- if that is the problem. John ./da-avx did not work, however ... Ubuntu is funny -- doesn't like giving access to root, etc (I may have to try a different distro, i.e. one that doesn't make root access really difficult) Nonetheless, I moved 'da-avx' to /usr/bin and ran 'da-avx with the following command line: da-avx --input=testinfile.wav --info=2 --fa --fgh2 --tone=-12.94 --basic --play The result: Audio "DHNRDS DA decoder" V1.2.2A 26Mar2020 -- Author John S. Dyson -- Copyright 2017,2018,2019,2020 License terms are in accompanying documentation Time Out: Required CPU type: LINUX: X86-64bit with AVX2: i4xxx or greater Affinity entirely disabled Input file: "testinfile.wav" Output file: "<stdout>" totaldelay: 814 execl: No such file or directory I am not sure if I am getting somewhere? mQa is dead! Link to comment
John Dyson Posted March 28, 2020 Author Share Posted March 28, 2020 Oh, you are getting exactly what I expect... I didn't send a copy of aoplay, which is needed for the --play switch. Instead of a play switch, just pipe the output to the SOX play command like this: da-avx --input=testinfile.wav --info=2 --fa --fgh2 --tone=-12.94 --basic | play -q - The above is EXACTLY how I use the decoder (except I usually start with a flac file like below.) The only time that I use 'aoplay' on Linux is for testing the code for Windows. It is much nicer to be in the habit of the play/sox command, for example if you want to filter the highs a bit, just to experoment, then you can do this: da-avx stuff | play -q - treble -3.0 9k 0.707q Also, if you want a flac output file, then you can do this: da-avx stuff | sox - outfile.flac (You know 'da-avx stuff' is whatever full da-avx command that you want to use.) If you want .flac input and output, then you can do this: sox infile.flac --type=wav - | da-avx stuff | sox - outfile.flac Cool, huh? SOX and da-avx mix really nicely... John Link to comment
John Dyson Posted March 28, 2020 Author Share Posted March 28, 2020 Instead of the 'short examples' in the docs, might have beren slightly out of date, etc. Here I am including two Linux command files, one is what I currently use for the EMI 48 Olivia singles, and the other is for 'Crime of the Century'. These are the exact commands, even though I had to slightly edit hte scripts because I use the scripts in an environment rather than alone. Also, these are BASH scripts, so only applicable to Linux or running Cygwin on Windows. * Use command or shell scripts for full-album decodes. Doing it by hand is too tedious. I am sometimes tempted to start with my full-album flac files instead of the chopped up ones that I normally listen to. * Most of the time, I use SOX to convert input and output files to .flac from the internal floating point wav files. Sometimes, like with Supertramp stuff, I create full quality floating point .wav files, then subsequenty convert to flac at different quality levels for different purposes. The scripts below might seem complex, but is a way of showing the 'automation' that I use to rebuild my library. Actually, I have a higher level shell script that I select the various material that I want to recompile. These have been carefully tweaked/tuned, so might look a little different than the examples in the documents -- but both versions work fine. Note the use of the really advanced mode of '--xppp=12 --xf=7' on 'Crime'. Actually, you might just want to use --normal mode for that -- the super high quality mode is VERY VERY slow. * The tweaks in the scripts result from an attempt to precisely undo the EQ used by the distributors, which is often slightly different than the defaults in the decoder. The WORST problem is usually how they muck up the stereo image, refer to the use of '--fw=none --wia=2.0 --woa=0.50 --wof=1.18', this is not needed most of the time, but the Supertramp stuff is very atypical. * This Supertramp example is worst case... Even though the script is Here is 'Crime': (Note this is exactly what I used to create the most recent V1.2.1F-14 verision, Serial number=1010 (should be imprinted on the flac files): # this creates a .wav file #!/bin/bash export SOX_OPTS="" export VER="V1.2.1F-14" mkdir -p "${VER}" for fn in /music/Supertramp/*Crime*/[0-1][0-9]*.flac do bn="$(basename "${fn}" .flac)" echo "${fn}": sox "${fn}" --type=wav - | \ ~/ap/nrs/dabuild/da-avx --outgain=2.0 --info=3 --fa --fgh1 --fhh1=13500 --mfm=4 --fw=none --wia=2.0 --woa=0.50 --wof=1.18 --fd --tone=-12.875 --xppp=12 --xf=7 --affinity --bextdesc="FA Supertramp, Crime Of the Century" --overwrite --output="${VER}"/"${bn}".wav done Here is 'Olivia': !/bin/bash DECODER=/home/jdyson/ap/nrs/dabuild/da-avx MODE="--xpp" INFOMODE="--info=1" DECODERARGS="--affinity" SOX_OPTS="" #OUTPUTRATE="rate -v 88.2k" #OUTPUTRATE="rate -v 48k" #OUTPUTRATE="rate -s -L -v 48k" OUTPUTRATE="" #OUTPUTCONTROL="gain -n -1.0" OUTPUTCONTROL="" mkdir CD2 for fn in /music/olivia/CD2/*.flac do bn="$(basename "${fn}" .flac)" outputfile="CD2/${bn}.flac" echo Decoding FROM: \"${fn}\" echo Output TO: \"${outputfile}\" sox "${fn}" --type=wav - | ${DECODER} ${DECODERARGS} ${INFOMODE} --fa=1 ${MODE} --tone=-12.905 | sox - "${outputfile}" ${OUTPUTRATE} ${OUTPUTCONTROL} done mkdir CD1 for fn in /music/olivia/CD1/*.flac do bn="$(basename "${fn}" .flac)" outputfile="CD1/${bn}.flac" echo Decoding FROM: \"${fn}\" echo Output TO: \"${outputfile}\" sox "${fn}" --type=wav - | ${DECODER} ${DECODERARGS} ${INFOMODE} --fa=1 ${MODE} --tone=-12.940 | sox - "${outputfile}" ${OUTPUTRATE} ${OUTPUTCONTROL} done Link to comment
John Dyson Posted March 28, 2020 Author Share Posted March 28, 2020 One more comment about using the DA/FA decoder: Here is my typical kind of command for flac input/output: sox infile.flac --type=wav - | da-avx --fa=2 --tone=-12.90 | sox - outfile.flac The command CAN be that simple. There are some gotchas though. One gotcha is that sometimes the output levels can sometimes go above 0dB (start to clip.) That can be fixed by decreasing the output gain, use the '--outgain=-6.0' for -6dB output gain. A 'pacifier' is sometimes nice, that is -- a progress indicator... I often use '--info=2', which gives a second by second log of the gain behaviors, but normal users are probably happier with a dot-progress indicator by adding '--info=1' to the command. The '--fa=2' is what says: 'run in feralA mode with filter #2'. There are 6 filters, but 5 are normally used. They are LPF, filter0 is 3kHz, filter1 is 4.5kHz, filter 2 is 6kHz, filter 3 is 9kHz and filter 4 is 12kHz. Usually, filter2, then filter1 then filter0, then filter3 are the best orders. For typical classical recordings, try --fa=classical, which uses a different configuration of stereo image handling and actually gives an HF boost instead of HF cut. If the HF boost isn't helpful, then use --fgh5=0 to turn off the HF boost. For higher quality, instead of the --basic switch (in the demos), the --normal switch (the default), you can try --xtra or even --xpp, which will clean up the audio, but doesn't filter the audio at all. Using fancy math, the distortion effects are reduced. Most audiophiles can probably notice the real distortion in --basic mode, which is actually BETTER than a true DolbyA. However, --xpp is really pretty good. The super, but very slow modes like '--xppp=12 --xf=7', remove virtually all modulation distortions, but can be unkind because the sometimes kind 'fuzziness' of DolbyA is totally obliterated. John Link to comment
John Dyson Posted March 28, 2020 Author Share Posted March 28, 2020 I haven't gotten up the energy to do a big write-up on decoding the FA stuff, because there are a lot of opportunity for optimization. The default settings are 'default' and generally sound okay. They are 'safe', but sometimes material can benefit from tweaking. If I get some requests, I can provide a more complete tutorial other than --fa=3 type mode setting. As an example, I can vastly clean up the ABBA or olivia stuff -- but even the basic settings are generally a major improvement. It is like going from a 'really good' result to a remaster, but not doing any real remastering. For example, there is a gain boost filter that is in the 2.5k though 3kHz range, and the Q can be 0.5, 0.707 or 0.8409. I am adding some built-in modes that set these parameters, but for now, all one can do is to change the settings oneself. It requires REALLY PERCEPTIVE hearing to figure out when the modified filters are needed... Using the DA can give really good commercial quality results, are MFSL-type-on-a-good-day results. I suspect that a lot of the problems even with remastered material nowadays is that the actual master tapes might not be easily available anymore (e.g. Universal fire), and so backing out previously EQed material might be needed. I noticed on the Carly Simon MFSL album that I have -- the DHNRDS can blow that album away, yet the MFSL IS properly mastered. It is ALMOST as if they aren't starting with pristine material. So, there are a lot of possible hints for professional level quality improvement -- my own limitation is my perception, not the usage skils. If someone really has good perceptive ability, it makes me wonder if there might be a legal/legit $$$ opportunity for them?!?!? No matter, in a few weeks, I'll be doing more documentation -- but the last few days, I have been a little 'slower' than usual. Sorry about the state of documentation, but if anyone does have a troublesome decode -- just show me an example, and I cna probably help fix the problem!!! John Link to comment
John Dyson Posted March 28, 2020 Author Share Posted March 28, 2020 Found a slightly better adjustment for normal FA use. I'll make these changes the default for FA mode decoding in the next release. These are minor, but do seem to improve certain natural vocals... Add the --fbh=2750 and --fbq=0.7071 switches to the command line. I'll be doing a new release in a day or so, and then these will be the default. This will really improve natural vocals. These are part of the settings that don't normally need to be changed often -- only need changing on specific recordings. Some day, I'll explain what these do, but suffice to say that these are part of the FA -> DolbyA EQ mechanism. John Link to comment
John Dyson Posted March 30, 2020 Author Share Posted March 30, 2020 I have been taking it easier in the last week or so, so ran the entire ABBA catalog -- the best album CDs that I could possibly find. I STILL couldn't get the pristine results that I wanted. The results are VERY good, but not perfect like I hope for... Just made another breakthrough, and I am not sure that this change is needed on ALL FA recordings or just a subset. I have been playing with the best version of the change -- trying to intuit (like I must) the best choices, and think that finally the smooth midrange is achieved. You know, on this FA effort, I have to reverse engineer and work with minimal information. A person must be careful about false or misguided information becoming a basis of both engineering and personal decisions. You cannot simply take a statement from someone as fact, but MUST verify and use best judgement. This is a philosophy that I now use through my entire life, developed from all of the mistakes I have made in the past... So -- just like my FA effort -- don't accept claims on face value, and in fact it makes no difference how many sources you use as information -- most likely the same biases come from the same source. USE your developed wisdom, and over this effort and other events in the world -- THINK, don't just assume that a claim is true -- even if from a previously credible source... I refer back to the Sony DolbyA patent -- you should NOT use that as a guide to make a DolbyA decoder!!! The result will be wrong, and even if the bugs are fixed, then the result will be mediocre!!! Not only that, the resulting design will likely be encumbered by the patent -- that thing is poison for any effort like the DA project... Trust, but verify... Then, don't really trust -- just use your own built-in wisdom and logic. If a statement is ALMOST true, but not quite reasonable -- it is probably totally wrong and is propaganda. In a way, the little twists and turns on this DA effort (and worse, FA effort) have been interesting. I have really learned something about industry politics, but I have been so very careful to keep myself and my project clean where it needs to be. I am not perfect in that regard, but the project is pretty much immune from any IP claims... At worst, a few unforseen tweaks might be needed, but absolutely no proprietary methods have been used in the decoder (except for IP that I actually own myself.) Even people who you might normally trust sometimes have reasons for misinforming.... John Link to comment
John Dyson Posted March 30, 2020 Author Share Posted March 30, 2020 Here is what happened that really weirded me out about decoding this FA stuff.... This is strange, and I am not sure exactly if this is planned or not... You know that on any fast gain control device, the lower frequencies can intermodulate with the higher frequencies, and if not properly handled creates a kind of distortion -- a kind of IMD. This IMD happens both because of gain control modulation but also on the edge of the gain control transitions.... I have been trying to get a smooth, clean sound out of these ABBA recordings, but there was always a vibration/veil on the recordings -- both before I decoded them and afterwards, but the sound is NOT something that would be in any normal recording. I found that there is some modulation in the 1kHz range, where the lower frequencies mess up the sound of the lower midrange (all mixed up in the 1kHz range.) The DHNRDS cannot create that kind of distortion all by itself, unless the frequency response of the system is not flat. The DHNRDS has special avoidance of MF IMD -- VERY SPECIAL... Makes me wonder if that avoidance might be a bad thing? This guess is probably wrong also though -- because the MF IMD mitigation isn't functional on the lower decoding modes, so that guess must be wrong... A small boost in the lower midrange before decoding, and then a compensatory cut after decoding seems to clean up the sound a little bit, but I hadn't found the 'null' in distortion until yesterday, and this null is profound. I TRULY wish I understood what is going on, but it is almost as if a pre-emphaiss/de-emphasis was done to corrupt decoding attempts. Note that the decoder is exquisitely flat all by itself, and the only reason for any deviations come from the EQ that I had to add. The deal is that I haven't beem meddling with the kind of response curves that it appears that are needed. That is, all of the midrange gain changes are such that there is a dip somewhere between 500Hz and 3kHz, but it is symmetrical where both sides of the dip have equal gains. I am finding that there is a 'key' set of compensatory boost/cut, where approx 0.75dB bass boost at 1kHz and 0.75dB bass boost at 250Hz, but then immediately after decoding, the equivalent cuts -- that ROUGHNESS is mostly gone. It is a very very specific set of filters, and if the frequencies vary too much, the the distortion null doesn't happen. *** ADD-ON: how come so much distortion reduction with only 1.5dB total gain change? WTF? Is this built-in the signal? These resulting ABBA decodes are incredibly superior, but more interesting are results from other recordings. The ABBA decodes in the highest quality mode still have about 2Hrs to go, and then it will be very interesting to check the other recordings to see if this EQ is exactly the same for those also... Really weird, almost metaphysical -- one of things that I do not like to ascribe in engineering. This is the reason why I have been a little weirded-out, and before this releae with a set of very noticeable improvements, I want to understand this potential change... I am hoping that this will be the last developmental DHNRDS FA change, and the rest will be bugfixes. John Link to comment
John Dyson Posted March 30, 2020 Author Share Posted March 30, 2020 Here is a before/after demo -- along with the lower MF distortion fix, this also has a MAJOR MAJOR MAJOR stereo image fix. The old version is the V1.2.2A version, the newer version is the V1.2.2C version. This fixes a LOT of really strange, difficult to reverse-engineer bugs!!! These snippets are short, and well worth comparing... Also, I found a simple example of the original CD 'woody' sound, and a properly decoded version of a recording (the olivia newton john song) -- note the woody, kind of almost nasal sound from the undecoded 'CD' version. The 'new' verison soudns more normal, and IS less compressed... John 08 - I'll Never Fall in Love Again-V1.2.2C.mp3 08 - I'll Never Fall in Love Again-V1.2.2A.mp3 (15) [Olivia Newton-John] Winterwood-CD.mp3 (15) [Olivia Newton-John] Winterwood-new.mp3 Link to comment
John Dyson Posted March 31, 2020 Author Share Posted March 31, 2020 I am doing some passive 'tweaking' -- really learning a lot of details about gain control... Fast gain control is exquistely EVIL. Since I have a little time on my hands, and have been trying to make further FINE progress beyond the previous steps. I admit they are still imperfect as they'll never be perfect, and even the taste might be more towards the FeralA sound -- and that is okay. My software was never intended to be for mass appeal, but rather a closer to accurate representation of the now 'ancient' music... However, I sure hope that a few people enjoy the results!!! :-). Anyway -- I have been trying to find what is going on with the latent distortion coming from the 250Hz to 1kHz region modulating itself and the higher frequencies (like up to 3-5kHz or so.) The possibility of the problem happening MAKES SENSE, because the main band for gain control is the approx 74Hz to 2880Hz frequency range, with very a low Q filter (about 0.470 for the 3kHz filter) allowing the 2880Hz side to reach up to 6kHz or more. The 74Hz size is problematical for its own reasons, but that isn't what I am dealing with right now. * The design is vulnerable to the latent distortion because of the 'fast gain control' and the impossibility of maintaing freq response so flat as to avoid all sources of distortion. This old NR stuff was a REAL devils bargain!!! Here is what I am hearing -- sometimes on complex choral singing, I am getting a modulation of the middle/upper registers of the voices. Of course ,you know what I am listening to on these tests -- ABBA. However, they make really good test material for this problem. This 'modulation', or actual inter-modulation between the lower midrange (250-1kHz) and the upper registers (2k-6kHz) sounds awful. This problem seems to happen on every copy of ABBA that I have, and when that isn't a strong effect, then the sound is muddy instead. There is NO good copy of the material. I found, after exhuaustive expermentation, that a certain VERY MILD LF boost before decoding, and then the equivalent VERY MILD LF cut after decoding does smooth out the sound. This confounds me because the DHNRDS is pretty much flat until about 100Hz, then it does have a 0.3 -> 0.6dB boost below that -- not a major problem. What the heck is going on? The original solution -- which produced the small demos previously posted, got rid of perhaps 1/3 to 1/2 of that ugly modulation. The boost for the first version was 0.75dB 1st order bass boost at 1k and 0.75dB 1st order bass boost at 250Hz. I knew that it wasn't a complete solution, but it helped. Also, it appeared to cause NO damage to any other material, including Supertramp and various classical pieces. Since I have plenty of time between sleeping, I thought I'd give some other filters a try, to see if there could be further improvement. Note also, the ideal values were nice, almost standard, ROUND NUMBERS in audio gain terms -- so something does appear to be intentional. The eventually pre-EQ and post-EQ filters ended up being, all 1st order, so no Q values are applicable: After a lot of initial tests: 1st guess that resulted in good, not perfect, initial results: 1000Hz, 0.75dB, 250, 0.75dB After a lot more tests: 2nd guess that resulted in significantly better, but still not perfect results: 1250Hz, 0.375dB; 1000Hz, 0.375dB, 750Hz, 0.375dB, 375Hz 0.375dB Of course, before decoding, these filters are 'boost', and after the decoding then the filters are 'cut'. Also, after finding these 4 filters, I tried adding filters onto the end -- at 1500Hz and at 187.5Hz, resulting in an increase in distortion.. These seem to produce a minimum of some kind, because any change seems to make the results worse. The reason why I claim these filters are in 'round numbers' is that the HZ values seem to be on a 250Hz boundary, and I have tried others for worse results. Also, 0.375dB is 1/4th of 1.5dB, which is a nice, round audio gain level. Of course, 1.5dB is 1/2 of the normal 3.0dB so often used for this or that. Oh well, in a way, it is nice not to feel hurried, and be able to really tweak and collect information. However, this does NOT explain why there is a benefit from this. The DHNRDS itself is very flat considering the math and filters going on -- within +-0.25dB or so from 100Hz on up to the top. There are a few issues below 100Hz, but not really that bad. The problems below 100Hz probably result from the coupling and transformers in the original DolbyA HW, and not fully emulatting all of the coupling and bypass capacitors in the electronics. As always, anything metaphysical like this can be disabled when using the DHNRDS. However, the default behavior will always be my best guess that provides the best performance. Please note, metaphysical things like this are NOT the norm in the decoder... I am writing about this because the matter is so exceptional. I can guess about what is going on, but it would be a 100% guess at this point. VERY FEW people in this world know what is going on here, and most have probably already left us. No matter what happens, this upcoming release will be better... I am being very careful about every detail that I can think about (and willing to take suggestions!!!) John Link to comment
lucretius Posted April 1, 2020 Share Posted April 1, 2020 On 3/28/2020 at 5:25 AM, John Dyson said: Oh, you are getting exactly what I expect... I didn't send a copy of aoplay, which is needed for the --play switch. Instead of a play switch, just pipe the output to the SOX play command like this: da-avx --input=testinfile.wav --info=2 --fa --fgh2 --tone=-12.94 --basic | play -q - The above is EXACTLY how I use the decoder (except I usually start with a flac file like below.) The only time that I use 'aoplay' on Linux is for testing the code for Windows. It is much nicer to be in the habit of the play/sox command, for example if you want to filter the highs a bit, just to experoment, then you can do this: da-avx stuff | play -q - treble -3.0 9k 0.707q Also, if you want a flac output file, then you can do this: da-avx stuff | sox - outfile.flac (You know 'da-avx stuff' is whatever full da-avx command that you want to use.) If you want .flac input and output, then you can do this: sox infile.flac --type=wav - | da-avx stuff | sox - outfile.flac Cool, huh? SOX and da-avx mix really nicely... John Sorry I didn't get back to you sooner. Although I can run 'play' (SoX) stand-alone in Ubuntu just fine, when I pipe the output to 'play' (SoX) via 'da-avx', it plays way too fast (like when the sound driver is broken). Any suggestions? OTH, this may be a problem on my end because I am running Ubuntu in a VM. [I'm going to install Fedora in a VM soon -- maybe that will work better?] In any case, I can run da-avx on Windows, just fine. John Dyson 1 mQa is dead! Link to comment
dean70 Posted April 1, 2020 Share Posted April 1, 2020 I have developed a PowerShell script dap.ps1 for Windows to make it easier to preview decoding. Copy dap.ps1 into folder where da-avx.exe resides. sox,exe and da-avx.exe should be both added to the system path. **Note dap.ps1 will play to the default audio device only. Select the DAC as the Windows active audio device first. To use: 1. open PowerShell prompt in folder where music files reside (Shift+right-click and select Open PowerShell window here) 2. enter the command dap.ps1 '<flac filename>' -da "<parameters to pass to da-avx.exe>". This will play decoded in real-time from the beginning of track. Press Ctrl+C to stop. --basic is already included in script, so no need to add. To play a limited section of the track , add -range "<start>,<duration>" option in mm:ss format. To play without decoding, remove the -da option and specify -nop. This will play directly from sox.exe. To play an a/b comparison, setup 'a' sample as above, then add -b to command and one of -bnop or -bda and -brange for 'b' track configuration. The window will display which section (a or b) is currently playing. dap.ps1 John Dyson 1 Alchemy Desktop http://www.origen.net.au/Alchemy/ Link to comment
John Dyson Posted April 1, 2020 Author Share Posted April 1, 2020 16 hours ago, lucretius said: Sorry I didn't get back to you sooner. Although I can run 'play' (SoX) stand-alone in Ubuntu just fine, when I pipe the output to 'play' (SoX) via 'da-avx', it plays way too fast (like when the sound driver is broken). Any suggestions? OTH, this may be a problem on my end because I am running Ubuntu in a VM. [I'm going to install Fedora in a VM soon -- maybe that will work better?] In any case, I can run da-avx on Windows, just fine. Hmmm about the play... Interesting because the da-avx provides a .wav file through the pipe, and it normally works (for me)... So, I will definitely research it and try to figure out what I did wrong. Most likely, I did make a mistake or assume something incorrect on Linux... I am guessing that the version of sox on Linux that you are using might be happier with a full specification of the file type/data rate. It is really easy for me to add the full specification to the 'secret' command line that I send sox :-). I will definitely look into it, and check out what I might be doing wrong on Linux. Thanks for the data point on Linux play though -- and I thought that was the 'easy' one :-). However, thanks for the good news on Windows!!! John Link to comment
John Dyson Posted April 1, 2020 Author Share Posted April 1, 2020 13 hours ago, dean70 said: I have developed a PowerShell script dap.ps1 for Windows to make it easier to preview decoding. Copy dap.ps1 into folder where da-avx.exe resides. sox,exe and da-avx.exe should be both added to the system path. **Note dap.ps1 will play to the default audio device only. Select the DAC as the Windows active audio device first. To use: 1. open PowerShell prompt in folder where music files reside (Shift+right-click and select Open PowerShell window here) 2. enter the command dap.ps1 '<flac filename>' -da "<parameters to pass to da-avx.exe>". This will play decoded in real-time from the beginning of track. Press Ctrl+C to stop. --basic is already included in script, so no need to add. To play a limited section of the track , add -range "<start>,<duration>" option in mm:ss format. To play without decoding, remove the -da option and specify -nop. This will play directly from sox.exe. To play an a/b comparison, setup 'a' sample as above, then add -b to command and one of -bnop or -bda and -brange for 'b' track configuration. The window will display which section (a or b) is currently playing. dap.ps1 2.32 kB · 1 download Thanks for this -- I am a total Windows numbskull (three stooges lingo), and postings like yours actually help me also. Because of a telemedicine requirement, today I had to re-installl skype on my laptop (where I do the Windows builds for DHNRDS), and it scares the h*ll out of me - you know, password recovery, etc. I just had to change all of my passwords, luckly I remember a very old password from 25yrs ago, and wont' forget that one :-). So, ANYTHING that anyone can do to help with Windows, including criticism (linux & windows, either one) -- I greatly respect and appreciate it... I'll still be disappeared for a few more days -- maybe dabbling... The testing results for the new version are 'smooth' and 'clean' so far. John Link to comment
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now