'FeralA' decoder -- free-to-use

[John Dyson]

Clairifcation of previous post -- I intend a new release, where I am totally focused on development and not doing any public test decodes (beyond single cuts) for approx 1 wk.  A neurological problem along with overwork, plus impatience on my part has encouraged several serious mistakes.

 

This next release will be SCRUBBED CLEAN against quality issues.

 

Contents of next release in about 1wk:

 

1)  Windows based file I/O on the Windows release (fixes some permissions issues)

2)  EASY two pass decoding, no rate conversion steps, etc...

3)  Scrub the code for residuals from my coding mistakes.  There were several cases where I had repeated lines of code -- IN THE DECODING ALGORITHM ITSELF --  because of the neurological memory gltiches.  This is what panicked me and forced the mea culpa in the previous post.

 

I am hoping that the FA decoder is wrapping up.  This two pass thing (and all of the precision issues associated with it) has delayed the completion by over 1 month.  The eventual advantage will include more precise single pass decoding.

 

Note that FA doesn't just compress the high frequencies by the DolbyA 10dB, but actually 20dB (or more.)   That is BAD and one reason why FA recordings just don't sound right.  (As you know, FA comprises almost all pop recordings, and a lot of classical, including some Telarc releases.)

 

I'll be posting from time to time, and keep everyone up to date on the next release -- with the improvements above.

 

I wish that there was no reason to do this project -- but the suckky quality of almost all CDs is intolerable.  Other than the distributors 'seeing the light', this decoder is the best-bet at having better quality recordings.

 

John

 

[R1200CL]

How to implement FA decoder into Roon or HQPlayer embedded ?

Is this something that can be done in the future ?

Maybe Sonore’s Sonicorbiter Linux can include it ? Do you se that as an option ?

 

Does the decoder need to be implemented before other DSP in a playback chain ?

 

[John Dyson]

On 7/25/2020 at 8:06 AM, R1200CL said:

How to implement FA decoder into Roon or HQPlayer embedded ?

Is this something that can be done in the future ?

Maybe Sonore’s Sonicorbiter Linux can include it ? Do you se that as an option ?

 

Does the decoder need to be implemented before other DSP in a playback chain ?

 

I have a simple loop that I am planning to document in a few weeks.   It is *really* simple if implemented in SIMD, and use the right C++ classes.  (need SIMD because of the 4 bands & 2 channels, therefore you have 8 variables at a time for calculations.)

A decoder similar to the quality of the old DolbyA HW can be done reasonably simply....  I'll see if I can put together a simple, stick-figure version in C++ (none of the quality improvements) in a few days.   The hard part is in some of the details, but the decoding loop is clean and I have found NO patents infringed.   Except for the bandsplitting code (tricky to do) and the very simple HF0/HF1 cancellation mechanism (about -- maybe 10 lines of code), the main loop is only 159lines with a few comment and blank lines.)   The SIMD support code requires 8 floating point values per variable, but I have some SIMD emulation routines that generalize this to non-SIMD machines (but would run 8x slower.)   Also, the code needs SIMD log and exp? functions (I forget the details.)  There are lots of little details, but I wouldnt' be surprised if the whole thing couldn't be implemented in 1000 to 1500 lines of SIMD C++.  (The full, super high quality decoder with Hilbert detectors and modulation distortion removal (along with command line, .wav file & multi-threading support) is approx 25k lines.)

 

* There is a DolbyA related patent owned by Sony -- it is extremely incomplete, and is not adapatble to the anti-MD anti-IMD capabilities.  The Sony patent goes into NO details about the attack/release, but the hard part of doing an accurate DolbyA decode is the attack/release calculation -- it needs to be super-precise and is very nonlinear -- simple constant RC timeconstant attack release need not apply.  Also, the attack/release is two layers -- like a slightly fast attack/release is for fast transients, and then a slower attack/release kicks in for longer events.  Also, the attack is carefully crafted to mitigate IMD from the detector, but has a window in the 1kHz range, and a liklihood of creating distortion in the 9kHz range if not tuned correctly.

 

Very little of the DHNRDS isn't SIMD.   (e.g. it really likes AVX2, SSE and I even compiled the code for Neon early on.  I haven't implemented AVX512 special case code yet, but would run more efficiently even without modifications.)   The code is very optimized for recent versions of Clang++, and uses the 2017 version of C++.   GCC runs about 30% slower -- it isn't as good at SIMD, and the Intel/Microsoft compilers don't get honorable mentions.   Implementing the code in C or other non-SIMD friendly languages would best be done by hand-compling the C++ original.

 

About the location in the playback chain -- it MUST be intimately connected to the digital recording, with absolutely no level modification or EQ.  It only works easily on un-normalized recordings, otherwise tweaking for the calibration level can be somewhere between tedious and difficult :-).

 

 

[John Dyson]

Wow -- maybe MAJOR breakthrough for FA decoding.  There was always a problem with the two layer decoding, but I couldn't figure it out -- kept improving the decoder accuracy, but something was wrong in some decodes...   Intuitively, there was also a problem with the calibration being the same for both passes.  That is beating REALLY hard on the signal when the calibration is in the same place.

 

Answer:  they use the previously described calibration levels for the last ENCODING pass, which means the first DECODING pass is the previously described calibration level.  The next calibration level for the second DECODING pass is 10dB lower.  So, if the normal first decoding pass is -13.50 for a given recording, then the next pass is -23.50.   Likewise, for the olivia stuff (where I will update the snippets and make them available here), it needs to start with -19.50 (approx), and so the next pass is -29.50.

 

This remedies the sense that sometimes there is too much expansion, and makes usage result in much more pleasant sound.   Previously, there was an improvement, but with this change, the improvement is more profound.

 

Working on it, and gonna build in the calibraiton offset for future users -- for the all-in one 2 pass mode.

 

Demo snippets coming soon.

 

John

 

[John Dyson]

Here is a notice that I sent to the PM group -- this follows up on the WOW mentioned earlier:

Another notice about the improvement in USING the decoder -- no changes internally...

Remember all of the worries about whether or not to widen the stereo image?   That is, the use of --wof=1.19?

 

When doing the staggered calibration levels, the problem of --wof=1.19 is GONE.   Apparently, when spreading the expansion (nominally 10dB in most audible ranges) all the way from -13.50 down to -23.50 then down to -33.50 (covered in two expansion ranges), the stereo image is CORRECTED!!!!   (This would be using -13.50 and -23.50, not just -13.50.  Some recordings are offset by 6dB, 10dB or apparently even 20dB?!?!?!)

 

This is a major question remedied.  Stereo image appears to be fully corrected.

 

This revelation by me, no time to listen to music!!!, is why we need more knowledgeable users.  The TECHNOLOGY IS CORRECT, this next version is DEAD ON correct, but we need MORE people with time and good ears and the willingness to invest some time.  This is for the good of recordings to the consumer, and I will only benefit by reputation (or undoing the damage that this might have had in some circles.)   This is a contribution that I cannot do all by myself, I have already burnt myself out mentally and physically a couple of times.  Just the decoder itself is an almost impossible effort, I need help in using the decoder.

 

Working hard, and just avoided another severe case of burnout.  But, we should have something amazing (moreso than before) with this multi-pass thing now working the way that it should.

 

This is a relief, and another tweak lost (final stereo image correction), but I need to determine that the 10dB offset is always (or generally always) true.

 

John

[John Dyson]

New, full quality *snippets* of the Olivia Physical album.  At least, in more complete versions of the songs, the cymbals are finally coming out as if they aren't sythesized!!!  Here are the snippets -- I find the 10dB offset very interesting (yet another part of the horrible 'encryption' of consumer recordings.)

 

https://www.dropbox.com/sh/4r3l6cctn7apvyi/AAC2OTYsbYQYqGaUmZBed9z9a?dl=0

[John Dyson]

Here is a very good example from a HIGH RES carpenters recording, and a decoded example from EXACTLY the same recording.

 

https://www.dropbox.com/sh/nx5nr848ln8bkqx/AAD-sXDajtq4SQAypWaPyU1Ia?dl=0

[John Dyson]

Release status:

I'll probably do an updated, vastly improved single pass release tomorrow.   The revelation about the calibration level staggering has thrown a monkey wrench into the  works.   I have been testing and verifying the hypothesis (so far true), that there is always a 10dB offset in the calibration levels.

 

I intend to build-in as much as possible for the 2 pass command line to make it as easy as possible.  However, the staggaring might have added another variable, but at least so far, it seems okay.

 

I suggest, unless you are actively using or want to use the decoder in 1 pass or DolbyA mode, wait until the 2 pass version is available.  It might take a few more days, because the staggering is something that I have to test for.

 

You can do two pass decodes manually, but it is a royal pain in the a**.   When the new 2 pass decoder comes out, it will include better logging of the DHNRDS commands into the metadata, and so many other nice features.

 

The niceest feature -- IT REALLY SOUNDS GOOD!!!

 

John

 

[John Dyson]

Crazy thing about almost all CD and downloads -- of course, we know about two passes.  I found some 3pass recordings, usually the highest quality/Hi Res recordings.

 

Apparently, the 'Industry' has a Russian doll scheme, where progressive layers of quality can be unveiled.   I wont' go into the technique or technology here -- most people reading this probably don't care about such details.  However, suffice to say, there is a definite layering scheme, perhaps a lot more primittive than MQA, but is still a layering.

 

Here is one of my first three (off a production line) of a three pass decode.  The others are also pretty good.  There is a minor adjustment needed for the three pass accuracy, but this is pretty good for the first full resuts that I have gotten, even for experiments!!!

 

Sorry that this must be a snippet, but this is WORTHWHILE as an existence proof!!!

 

John

 

03-Superstar-snippet.mp3

[John Dyson]

There is a new release with the quality improvements that support ENOUGH QUALITY for three level decodes (whoops 4 level, believe it or not).

 

IMPORTANT THOUGH -- on this version, multi-level decodes must be done manually, and therefore not practical unless you are very tenacious.  I truly suggest using this version for technical curiousty ONLY.   The new multi-layer/level decoder will do much more and sound even nicer.  The upcoming version will peel all of the 'Russian doll' layers, doing some things automatically, efficiently and with maximum quality (no file/rate translations other than in and out.)

 

For the new version, I will be producing a usable document, very simple, minimal confusion that shows how to do multi-layer decodes, but again BUILT-IN MULTI-LAYER  IS NOT IN THIS VERSION.   I have had to learn how to do the multi-layer decodes for myself, so I can show others.  This has taken a significant amount of time to figure it all out.

 

Honestly, I did this release because I follow through with promises if at all possible.  However, the multi-layer breakthroughs caused serious delays and required LOTS of testing.   I mean, some of the ABBA stuff is FOUR LAYERS!!!  CRAZY!!!  Most people seem to be tired of my ABBA demos, so I'll show a 4 layer result upon request (they really can sound better :-)).  Still, they sound like ABBA :-).

 

There is a several line user guide (included at the end also), and it is really accurate -- you really don't need much more to get started.  I'll repeat it at the end of this posting, but also there is a copy "quickusage-1.5.0B.txt" on the site.  All of the chaffe is gone from the doc, but ALL OF THE PREVIOUS FEATURES STILL EXIST.   I am keeping everything until I am totally sure that they are/are-not needed.

 

The decoder version is V1.5.0B, and the filename is:  "da-release1.5.0B-win-2020-07-31.zip".

The Linux version is: da-V1.5.0B-Linux.txz

The location is: https://www.dropbox.com/sh/1srzzih0qoi1k4l/AAAMNIQ47AzBe1TubxJutJADa?dl=0

 

The quickusage guide is also below -- I have also attached a snippet of a 3 layer decode of Herb Alpert's "Spanish Flea"  Two or three layer decoding on most (about half and half 2 and 3) recordings is producing the sound that I have wanted all along.  (Again, this release is more of a technical curiousity, but will work more perfect than any previous versions.)   The snippet probably generally sounds as good as the typical WELL MASTERED material, and doesn't have the foggyness.

 

* ALL PREVIOUS DEMOS WERE DONE WITHOUT SEVERAL IMPORTANT BUGFIXES.  THE RESULTS ARE AUDIBLY PERFECT IN MANY CASES (probably most.)

 

(remember the Spanish Flea snippet attached.)

--------------------------------------------

 

this is a super simple usage guide for DHNRDS FA V1.5.0B

 

VERY VERY VERY pre- preliminary, just a quick guide.

The only switches that you need are:

--input=infile.wav
--overwrite --output=outfile.wav

--tone=-13.55 (just use the default nowadays -- you can use tone if you need)

EQ style, use one and only one of the following:  --fcd, --fcf, --fcd=G, --fcf=G
(there are more EQ options, but for single pass decoding, the above are sufficient.)

Most material can be cleaned-up a little by doing a single pass decode,
but most material needs two or three passes (some ABBA needs four!!!) for full quality.

 

For decoding progress:
--info=0 (no progress)
--info=1 (dots)
--info=2 (gains)

 

For decoding quality, not as important because multi-pass decodes are more important:
--normal (default)  Much better than DolbyA quality
--fx (good improvement) Adds some MD scrubbing.
--fz (better MD scrubbing.)

The --fx and --fz commands do have options.  Don't worry about those right now.

 

Useful command lines are thus:

da-avx --fcf --info=2 --input=infile.wav --overwrite --output=outfile.wav
da-avx --fcd --info=2 --input=infile.wav --overwrite --output=outfile.wav
da-avx --fcd=G --info=1 --input=infile.wav --overwrite --output=outfile.wav

Multi pass will be coming in days, and you'll get amazingly better results.

 

John

 

 

 

04.Spanish Flea-snippet.flac

[John Dyson]

Since there are apparently some people using the decoder, and I won't be able to get the multi-pass done until at least near the end of week, I plan to do a new release sometime tomorrow afternoon.   The improvements are BY FAR more than desirable.  (normal mode decodes somtimes sound significantly better than higher quality-mode decodes on the current release.

 

I am testing the updated version right now, and making sure that no new bugs have been introduced.

 

John

 

[John Dyson]

Want to give an update about my attempts to decode the 'Al Stewart' recordings.   The problem is that I have found that they encoded it with 11 passes of DolbyA compression (or equivalent.)   The decoder works well, but not THAT well.

 

I did successfully run 11 passes, and the results came out, but kind of poor.  The CD is one really messed up recording.

 

At least, I know what the problem is.   Most common recordings can be improved by three or four levels of decoding.

 

John

 

[John Dyson]

Final (for a while) SINGLE PASS RELEASE.

Version:  V1.5.1A

 

This seems to sound as good as possible.  It works very well even when presented lower quality material, calibration errors, etc.

This does wonderful single pass decoding, but for your own sanity, for multi-pass decoding, wait until the release later on this week, or more likely early next week.  I'll try to get it done before the weekend, but I don't want to compromise quality or risk mistakes.  If you want to do multi-pass recording NOW, I will certainly be willing and happy to help you.  However, much of what you'll learn for the temporary multi-pass decoding will be wasted, because more automatic stuff will be done in the multi-level version.

 

There is a new, short doc called 'quickusage-V1.5.1A.txt', and it has been modified from the previous, with my revelations about decoding methods.  I am learning to use the decoder, only a few days earlier than you.  But, anything that I learn and can express, I will share it with you.

 

Look for the V1.5.1 files at the link below.  Also, as usual, there is a Linux version.  The fact is, I primarily test the Linux da-avx version, so that is the flagship.

 

After this release, I have a few minor possible speed/quality improvements to the base decoder -- purpose is to improve what I can improve before doing the multi-pass decoder.  it will be important to make sure that the single-pass decoder is "perfect" before starting the multi-pass.  I am not good at 'multitasking', so I cannot distract myself to work on sound quality problems when I am doing the complex multi-threading and messaging.

 

Here it is -- AND HAVE FUN!!!!

https://www.dropbox.com/sh/1srzzih0qoi1k4l/AAAMNIQ47AzBe1TubxJutJADa?dl=0

 

John

 

[John Dyson]

Realistic schedule for the multi-layer/multi-pass release will probably be after the weekend (after 10 Aug 2020, probably 14 Aug 2020), just because there are so many things to do.   Honestly, I have still been finding little decoding improvements, so that the high quality (anti-MD) modes would be needed less often.   I am trying ot make the --fx and --fz (esp --fz) be more appropriate for a research project, or the very very highest quality product.   For any of my own casual listening, the --normal mode that I am using right now (not yet released) is probably good enough.

 

* I am going to start calling the scheme 'multi layer' and not 'multi pass'.  Multi-layer doesn't create the notion of 'reprocessing'.  I like it more than 'multi pass', which almost sounds like a 'redo'.

* Realistic expectations should consider that a lot of over-compressed recordings are damaged beyond recovery.   An expander has limits as to what it can do, and if the compression is too extreme, the little hints that the expander (decoder) uses become obscured or even obliterated.

 

Some of the new quality enhancements are enabled by different nonlinear filtering, that helps to make the decoder avoid creating MD/IMD products as strongly.   I just tested the earlier --normal modes and the method close to HW device quality and the difference is so profound that the earlier performance is almost embarassing, and the HW quality is not worth using to decode the material.   The new  --normal mode is lightyears beyond the older stuff.

 

There are some new modifications which, through different means, avoid the creation of much of the MD.  Anti-MD still improves clarity, but sometimes can be somewhat tricky to figure out which is correct (using anti-MD or not.)   Also, the front-end gain control Hilbert transform filters have been improved so that they require a lot less CPU.  This is especially important when running 7 passes.   The input bandpass, and the Hilbert & Attack/release calculations need to be repeated for every layer/pass.  However, the anti-MD and final gain control will only be needed once.   The up-to-7 improved Hilbert transforms for the gain control/level calculations will make a noticeable speed improvement.

 

So, I can reasonably suggest that there will be a multi-pass release before 15 Aug 2020, most likely middle/late afternoon Fri 14 Aug2020. This is a realistic schedule that won't rush me into making mistakes.   I really do like to test the program as well as I can.  The basic decoding performance issues are starting to be 2nd order bugs, which make the multi-layer decoding more important.

 

Any time anyone wants a test version that corresponds to what I am currently testing & working on -- just tell me, it would only take me 20minutes to create a quickie release.  For Linux, I can just send the binary that I am testing right now.  The caveat is that a 'newer' test version isn't necessarily better than the previous release, but just might be a LOT better.  You can never tell.

 

John

 

[John Dyson]

Progress report:

The multi-layer decoder is moving forward pretty well. Still expecting a Friday early evening for the release.   The quality is audibly better, and the speed can be double that of the separate commands.  Listen to the stable vocal chorus -- the chorus with Linda's recordings can be tricky -- but zero problem with this decoder.    Also, the command line is MUCH MUCH MUCH simpler for multiple layer as below:

 

 ${DA}  --info=2  --fz=max --fcd  --tone=-39.55365 --ln  --fa=G  --ln  --fcd  --ln --fcd=G  --overwrite --output="output.wav --input="input.wav"

 

Basically, the above command does a four layer decode.  Each '--ln' switch starts a new layer of decoding.   Usually, there will be an 'FeralA initiator' after the '--ln' (layer next) switch, but not needed if same as the previous one.   The starting tone= value is usually based on -13.55365, and is usually a 10dB increment lower, depending on how many layers.   Sometimes (as in the example above), there is an additional -3dB or -6dB offset for headroom.   once the -13.xx or --19.xx tone= value is reached, then additional --ln layers wrap back around for additional expansion.

 

This expander is doing a LOT of dynamic range expansion, and would sound so 'normal' if it wasn't very precisely matching the compression originally applied to the recording.

 

The above command is exactly the form for decoding this Linda Ronstadt example.   The original CD was too muffled, too full, but not woody like some recordings.   Note that the in the decoded version, her voice comes through much better, and the background is more clean.

 

This is still preliminary -- but I believe A LOT better than before.   Most important, it is much easier to use, and no loss of precision at all between decoding steps.   The result will ONLY get better before Friday, but already it is pretty good.

 

John

 

12. Tracks Of My Tears-CD.flac 12. Tracks Of My Tears-1-decoded.flac

[John Dyson]

Super interesting news -- I just found a true DolbyA layer in many recordings.  This means it is very likely that the recordings were NEVER decoded to begin with during mastering.   This also increases thie likelihood of reaching the two track mastering level.

 

John

 

[John Dyson]

The release is very close.   There is now support for a final DolbyA layer, and the improvement is another big jump.   I think that the final DolbyA is one reason why the results were still sometimes 'squeezed' sounding.

Since the last step is DolbyA, it is very possible that we are reaching down to the distribution master tape.

 

Here is a test example, snippet from 'Classical Gas' -- the decoded version probably sounds better than you have EVER heard -- I have never heard of version of Classical Gas that wasn't terribly over compressed.   There is still some peak limiting, and seems like it was an artistic choice.   This is a 7 layer decode.   Decoding information in in the decoded snippet.

 

Listen to both.

John

 

Classical-Gas-decoded-snippet.flac Classical-Gas-CD-snippet.flac

[John Dyson]

The full multi-layer version is now available.  This is a massive upgrade in quality and capability.

V1.5.7A...  The Usage document is attached, but is also in the repository.  I am assuming that you have previously installed the program -- or take a look at earlier postings on how to install the software.

 

Multi-layer decoding is available without any complex command line stuff, and the decoding quality without the anti-MD is significantly improved  to nearly anti-MD quality levels.   The main reason for the quality rewrite/redo is because multi-layer decoding can take a lot of time when the anti-MD is fully enabled.   So, I worked on the algorithms to more carefully shape the attack/release waveforms, therefor further mitigating the 'fog'.   The anti-MD still results in superior quality, but not really worth it much of the time, even fr professional applications.   *--tone= choice is now relegated to two general choices:  for three layer decodes, either -39+N or -33+N.

 

Attached is a usage guide -- it is REALLY self explanatory, and decoding is very simple now.  The complex choices of modes is no longer needed.   Here are several decoding examples, most likely one of these will work for your application.  The choice is dependent on how intense the highs are.  You'll have to choose ad-hoc/randomly until I can better document the methods.  Sorry about the ad-hoc nature, but for now, it is best just to try the software.   Most/all of the previous features still exist, even the old FA initiators.   Basically, all that you should ever need to use now are:  --fcd, --fcd=G, --fa, --fa=G...   There is also --fcd=classical, --fcd=classical,G, --fa=classical, etc...

 

---------------------

 

Important to understand is  when the '=G' is used with the '--fcd'.  Needing '=G'  means that the highs are too strong and need to be rolled off.   The character of how the highs are rolled off depends on the location of the '=G' usage.   Here are commands that most likely can decode almost any pop material, if it is multi-layer encoded.  These are THREE LAYER decodes, which are enough for most material.  Sometimes 4 layers or starting with -49+N might be needed.

 

Generic command: (sometimes --tone=-33+N below instead.)

da-avx --input=infile.wav --overwrite --output=outfile.wav --info=1 --fcd --tone=-39+N --next --fcd --next --fcd

* The above command is most likely the best choice (other than for classical), and I'd guess at least 50% of my pop stuff matches it.

 

Possible variations for material that needs more EQ:

da-avx --input=infile.wav --overwrite --output=outfile.wav --info=1 --fcd=G --tone=-39+N --next --fcd --next --fcd

da-avx --input=infile.wav --overwrite --output=outfile.wav --info=1 --fcd --tone=-39+N --next --fcd=G --next --fcd

da-avx --input=infile.wav --overwrite --output=outfile.wav --info=1 --fcd=G --tone=-39+N --next --fcd=G --next --fcd

da-avx --input=infile.wav --overwrite --output=outfile.wav --info=1 --fcd=G --tone=-39+N --next --fcd=G --next --fcd=G

 

For classical material and certain pop material, you might need to use classical mode...   The generic command for using classical mode follows, and one of the 'variant' modes is also shown for the 'classical' mode:

 

da-avx --input=infile.wav --overwrite --output=outfile.wav --info=1 --fcd=classical --tone=-39+N --next --fcd --next --fcd

 

First, most likely, variant:

da-avx --input=infile.wav --overwrite --output=outfile.wav --info=1 --fcd=classical,G --tone=-39+N --next --fcd --next --fcd

 

Notice that when using 'classical' mode, it only needs to be selected on the first '--fcd' FA initiator.   Any 'G' that is needed is just tacked onto the 'classical' with a 'comma', like 'classical,G'.

 

It is also possible that the '--fa' variant is needed in lieu of the '--fcd' variant'.  The '--fa' will be useful when using the '--fcd' cuts the highs too much.

 

HERE IT IS!!! V1.5.7A

https://www.dropbox.com/sh/1srzzih0qoi1k4l/AAAMNIQ47AzBe1TubxJutJADa?dl=0

 

(Linux version is also on the repo.)

 

John

 

 

Usage-V1.5.6.pdf

[John Dyson]

Here are some example multi  layer decoding sequences.   Not all are three layers, but these are the 'ideal' as far as I can tell.  All of these below can be done in three layer.  Also, sometimes when there is a sequence other than the 'generic' as described in previous post, the material will sound 'okay' with the 'generic' setting.

 

There is room for adjustment, but the 'tweaks' are much more manageable than before.   One big reason for the previous need to 'tweak' the --tone= values was that the multi-layers was interfering severely with the single layer decoding.  This caused the single layer decode to be a compromise, but often still an improvement.

 

Once we found the 'secret' of multi-layer, then the --tone= values stablized very precisely.   Here are some examples -- just tested/verified by me:

 

 

Best of Bread
--tone=-33+N
--fcd --next --fcd --next --fcd

 

Linda Ronstadt's 1977 album
--tone=-33+N
--fa=G --next --fcd=G --next --fcd

 

Carpenters 1971
--tone=-39+N
--fcd --next --fcd=G --next --fcd

 

Carpenters 1970 (4 layer)
--tone=-39+N
--fcd --next --fcd --next --fcd --next --fcd

 

Carpenters 1973 (4 layer)
--tone=-39+N
--fcd --next --fcd=G --next --fcd --next --fcd=G

 

The Nat King Cole Story
--tone=-39+N
--fcd --next --fcd --next --fcd

 

The Beatles 2009 remaster (5 layer)
--tone=-39+N
--fcd --next --fcd=G --next --fcd --next --fcd=G --next --fcd

 

Carly Simon (greatest hits)
--tone=-39+N
--fcd --next --fcd=G --next --fcd

 

Herb Alpert & Tijuana Brass
--tone=-39+N
--fa=G --next --fcd=G --next --fcd

 

John

 

 

 

~                                                                                                                                    
~                                                                                                                                    
~                                                                                                                                    
~                                                                                                                                    
~                                                                                                                                    
~                                                                                                                                    

 

 

[John Dyson]

Heads up -- found a bug -- minor.  Just use the decoder with CD sample rate or 48k sample rate inputs.  88.2k and worse yet, 96k will have an incorrect LF level detection.  The filters tap counts were chosen wrongly,  with just enough error to be a problem.  I'l do another release tomorrow.  Just use CD files for input.  High-res would have to be down-converted to 48k or 66.15k for proper sound.

 

The error isn't deadly, just sounds wrong at 96k input..

 

John

 

[John Dyson]

38 minutes ago, John Dyson said:

Heads up -- found a bug -- minor.  Just use the decoder with CD sample rate or 48k sample rate inputs.  88.2k and worse yet, 96k will have an incorrect LF level detection.  The filters tap counts were chosen wrongly,  with just enough error to be a problem.  I'l do another release tomorrow.  Just use CD files for input.  High-res would have to be down-converted to 48k or 66.15k for proper sound.

 

The error isn't deadly, just sounds wrong at 96k input..

 

John

 

Bug fixed -- version is now V1.5.7B.   Very simple, minor difference.   Checked it out, now 96k should be okay.

 

John

 

[John Dyson]

Documentation update (not code update);

I added some notes to the Usage-1.5.7.pdf document, which mentions that sometimes '--fcf' is sometimes better than '--fcd=G', even though they are very similar.   '--fcf' is a single curve starting at the CD rolloff frequency, but doesn't stop.  The;'--fcd' is true CD EQ, but the 'G' adds a rolloff at 9kHz.   There is a slight 'hitch' in the curve when using '--fcd=G', while '--fcf' is more of a smooth roloff.  Each appears to have its valid usage.

 

I have already created some examples on the PM group, but will do some snippets for public demo some time tomorrow or soon thereafter.

 

John

 

Usage-V1.5.7.pdf

[John Dyson]

Final (fully working) release being tested.  Everything that I use for testing in my archives is being tested right now.  This might take another day or so.  (I have NEVER done a full test on the full testing archive all at once.)  The sound (when decoding is done correctly) is superior and more accurate.   Even the most difficult dynamics tests (the Telarc stuff isn't as severely compressed as  the hyper compressed Al Stewart recordings) are sounding MUCH better, and might even be almost correct to the pre-FA state.

 

Quality issues -- I finally had to fix the bass -- I had preferred a little less bass with more of the FA sound, but I decided to fully correct the bass.  Do NOT expect any of the 'woody' middle range that everyone is used to. (I might add a switch at the last minute to give the satisfying fake-midrange woody sound.)  ( Can you imagine any recording REALLY being made to sound like the woody FA sound?)

Everything is now based on 1st order filters (therefore producing the more correct, but REAL sound when using more than 3 layers.   This means that your speakers dynamics can sometimes REALLY be tested with REAL bass.)   This doesn't mean BOOMY bass, but more REAL bass on good POP material.  (Remember concerts, how the sound really sounds -- the recordings are less extreme, but much more dynamic than FA.)

*  I think of the woody 'FA' midrange similar to the 'ta-da' after a lot of classical recordings...  You know -- telling everyone that the song ends?   The woody midrange appears to be another of that kind of pacifier, but I decided that correctness is more important at this point.

 

There WAS some drift upwards in level, which made some decoding seem to allow more layers -- but it was wrong.   Expect to lose about 10-13dB in level after 4 layers or so -- this is because DA is a *downward* expander, so will naturally lose signal level after even one layer.   For normal listening with speakers, 3 layers is usually enough, because most speakers don't seem to be as good at dynamics as good headphones -- but if you go all the way to 4 layers (or more -- like the full 9, like on highly compressed materil -- my copies of Al Stewart collection), you will get more drift...   If material is highly compressed, then more than 4 layers can be really helpful, but with some tradeoffs.   Even though my Al Stewart collection can withstand over 8-9 layers (very very slow to decode), my ABBA stuff like 'Gold' only likes about 4 layers.  ABBA compression isn't all FA.  " Waterloo does have more FA layers, but those are mixing, not distribution.

 

* When doing testing/debugging of the new/actually correct midrange code, even a small sub-dB erroron each layer creates profound defects.  Each layer can compound defects.   At 5 layers, 1dB error somewhere in the spectrum can sometimes cause a 5dB error (extreme) somewhere in the spectrum.  I made some experiments where there was a 2-3dB error -- the results were crazy bad.  When loosing/gaining 8-10dB of lower midrange, the result is ludcriously bad.  The decoder has to be VERY accurate.  The code is NOT like a tone control.

 

Internal changes -- I had to decrease the amount of attack/release shaping (it is based on limited dB/second and not time constants) because more than a few layers otherwise caused a chunky/congested sound like I got earlier on the Beatles stuff.   I dont think that this attack/release speed-up has increased the fog all that much.  (The use of dB/second is contrary to the timeconstant approach -- but R Dolby, probably unknowingly, had designed some of the attack/release layer stuff based on more of a dB-linear approach, even though the HW design doesnt superficially look like it.)

LOTS of code has been removed. Will probably remove more right after these tests (proof that the old code will never be needed.)

====================================================

General trend in the code:

 

* The FA initiators will only be:  --fcd, --fcd=G, --fa, --fa=G, --fcf, --fcf=G  (I think that I have listed them all)   The old initiators were closer and closer approximations to the correct ones.  '--fcd' isn't really CD, even though I believed it to be for a long time.  A lot of the error came from using the sharper 2nd order EQ.   I got locked into 2nd order because of the poor support that SoX has for 1st order, and that was purely my own ignorance.

NO ONE has helped me reverse engineer this, except those KIND people giving feedback.  NO ONE in the industry has given away any 'secrets'   This has been long and difficult because NO-ONE has given me any hints (other than claiming it is wrong, or sounds bad, or pumping*.)  *I knew that they were lying about pumping -- I am the most sensitive person in the world about 'pumping' and gating (along with IMD & MD), even though I am bad at response balance.   There has been some distraction, misleading, and non helpfulness by some in the pro community trying to keep 'trade secrets' from being released to the consumer.  This would have been EASY if there was a slight amount of help and HELPFUL guidance by pros-who-know.

 

* The submodes (incl 'G' above):  G, A-F, I, J (I think H), plus the additional h, n modifiers of A-F

 

* All of the DA commands (mostly just thresholds/levels).

 

* For the --tone=, the '+N' extension will be included -- this eliminates needing to know all of the extra digits.

 

* The new metadata all included.

 

* Up to 9 layers is now functional, but best to use a max of 5 layers because of frequency response drift.

 

* The 'quality modes' include:  (none), --final (a little slower, not a lot better), --fx (moderately slow), --fz (slow), --fz=max (really slow -- forever)   You would probalby never worry about anything more than --fx.  However, --fz=max is almost the antithesis of the FA sound -- REALLY.

 

* the --pe switches are still supported. (last step EQ, trimming bass, control edginess, or maybe even bring up vocal range by a dB or so.)

 

* The useful switches like the stereo image (--wof, --wia,  --woa), --ingain, --outgain, --floatout, --intout, and channel switch --chsw still exist.

 

The old LF modifiers are gone also (sometimes needed to correct MF tracking because of my use of the  more mellow MF sound, 2nd order filters -- but no longer needed with the new lower MF EQ).    The decoder can track even the most difficult tests now (it always COULD HAVE SUPPORTED IT, but I got complaints about the lows not sounding right -- but the problem is that some people expect still expected the woody MF of FA.)   Now, EVERYTHING, all of my most extreme recordings can be tracked. (The spot checks say so, but I will be double checking in the mass-decode going on right now.)

 

I am running every Album in my test collection, plus a few more recently contributed as tests.  As soon as I get done running the 'contributed' tests, I'll remove them as they should be.

 

My own tests are intended to verify correctness, and NOT mastered sound.  My own ham-handed, but well intentioned mastering is almost always defective.   I am ONLY testing the correctness of the decoder.   The final version will be released ONLY when every recording results in correct decoding.   It is definitely delayed relative to my previous release goals, but it is time for this thing to be completed.

Every test that has run last night has been 'correct' so far.   The recordings now are in 'tweak mode' (e.g. trim some bass.)  *Note that the resulting, decoded bass is NOT excess, it is just that real POP bass is sometimes fairly strong instead of very compressed like on consumer FA recordings.  Nothing like the extreme loudness in concerts, but still profound.  Previous versions of the decoder did produce a little less natural bass than the version tested right now.   This makes tracking on the bass side a little more critical, but that IS as expected.

 

So far, it looks like the code passes.  I JUST MIGHT add back in the 'one-quarter woody' midrange code as an option, and make an option, but right now I am testing CORRECTNESS.

 

The decoder is being tested in '--fx', '--fz' and '--fz=max' along with the default '--normal' modes depending on recording.  Again, so far, after about 12hrs (seems like 12yrs), spot checks of the material, some of the most difficult, show NO defects.

 

John

 

[John Dyson]

This source code snippet is as promised.   This is a gain calculation loop source code copied out of the next release.   The decoder (FA and DA) is now very very correct.  This disclosed code is PURELY DA level, so has been solid for a long time -- this is used internally after each FA EQ stage.  The only relatively recent changes are some of the parameters WRT the nonlinear 'diode' exponential curves.  I am never 100% satisfied, and during the short times my hearing is working well, the values are double checked, and have recently only be tweaked with very narrow bounds (on the order of 1%.)

 

Even though this is a critical part of the code, it is only about 700 lines out of the 30000 lines (yes, I just measured them),  there is a lot more than just decoding DA, but also needs command lines, FA equalization, .wav file I/O & metadata, bandsplittiing, etc.   There is also a vast library of FIR, IIR filter support (dynamically based on current running sample rates), and massive transcendental and vector support.

 

This is only to show a basic algorithm (for those expert in reading C++/SIMD code), and there is a lot of subroutine support needed (e.g. dB2lin, vector exponential code, defining the vectors themselves, etc.)   The algorithm could be expanded into code using normal variables, but it would be MUCH slower.

 

If a decoder is written using this routine itself, and minimal I/O and bandsplitting code is used, it could be perhaps 2X faster than --normal mode on my decoder.

 

(I am still testing the FA decoder -- my testing is very aggressive, and know what it should/should not sound like.  I am not claiming that my test results are for good listening, but are intended to verify the correctness.   'Good listening' requires 'Good mastering' and 'Good decoding choices', which I am not good at. :-)).

 

I am verifying almost every recording that I have previously used for testing.  So far, 100% success (other than some botched decoding parameters -- as  usual.)

 

John

 

decoderloopsrc.zip

[John Dyson]

Still testing -- had to do a restart because of a bad transcription of some parameters in the decoder.  (yes, I don't always use cut-and-paste.)   The problem was with the corrected bass code, and continued to use a mistaken 750,1000,1250 Hz sequence  instead of the correct 500,750,1000Hz sequence.   I knew that the 750Hz centered sequence was correct -- but I had overlooked it.   Didn't detect the problem until listening, and finding a 'tight' and overly non-woody sound to the bass.   Strangely, my acid test case for bass (Some Phil Collins very best recordings require PERFECT Bass tracking) almost worked with the 250 error.   When I finally got worried about a consistent sound error (my hearing adapts quickly, so I didn't hear it until an overnight rest), I took a look and said to myself 'oh sh*t', and have to do a redo.

 

Sunday night at the very earliest for the release.  I am NOT letting any nit pass by me.

 

About the change to the correct 1st order LF EQ:

 

Basically, the 2nd order scheme was based more on a 1000Hz center rolloff frequency instead of the 750Hz center needed by 1st order EQ.   It is kind of strange, but the different order of EQ ALMOST works, but strange things happen.   I do believe that the 2nd order sound might be more tolerated when hearing FA recordings all of the time, but the 1st order sound is closer to what I remember during listening to real recordings, instruments and concerts.

 

Attached is a hellishly difficult section of a Phil Collins song (04 -- In the air tonight) from the 'Very Best' album.   Start listening about 17 seconds into this cut, where the background starts being significantly modulated by the forground percussion.   It only takes very slight errors for this slight 'jerk' to be severe -- it is very easy to much much more severe than the example below, which I am actually very proud of.  (I know, not impressive to the purists, but you are probably hearing a 0.5dB dynamic tracking error vs. time.)   To me, with 4 layers of decoding, this is a success -- sometimes perfection is impossible!!!

 

John

 

LF-tracking-demo.flac