Jump to content
IGNORED

'FeralA' decoder -- free-to-use


Recommended Posts

Just decided to update the decoder with the slight improvements that I mentioned.

 

First, the slight improvement in clarity seems to be worth doing right now.   Also, there was approx 2dB peak at 20kHz that I missed.

(The HF EQ sequence is a series of EQ steps, and the last EQ step was chosen to be -3dB@18kHz instead of the more appropriate -6dB...  So, the excess HF is NOT caused by a need to refactor, but is an end condition in an EQ sequence...   The changed parameter controls one of the matrix equalizers.)  I really don't know why I chose -6dB other as an experiment, because EQ rates are seldom mixed in such EQ sequences, and the shape is -6dB per step for that sequence.  (There is one very important place where there is a mix of -6dB and -3dB, but the -3dB is an exception, and it is appropriate considering the context.

 

Second, there was a slight variance against what DolbyA units do.  That has been fixed now, but the difference in sound is very slight.

The biggest difference is a slight efficiency improvement while doing DolbyA slightly more accurately.   The change makes the larger scale shape of the attack/release more compliant, but is difficult to hear on my purpose built subjective tests.   Most other historical differences were much greater.   This variance has been bugging me for about 1yr, and the bug  should never have an obvious negative effect -- real DolbyA units have so much fog that the FA/DA decoder is much more clean and can show the variance, while a DolbyA HW unit wouldn't show a change.

 

I found another speed problem, and this one is going to require some work.   There are some serious cache locality problems beyond the anti-MD sections.   If anti-MD is sped up by a factor of 10, there would still be only 20% real-time improvement.   A better way of organizing memory would be appropriate, but there is a lot of memory being moved around in the decoder.   The cache locality issues have caused 50-100 cycle CPU stalls on certain materials.   Of course, the CPU is sent off to run something else during that memory wait, but it still is a real-time delay that is in a critical path.

 

So, the new release (in the V4.8.3 series) will have slight speed improvements and be very noticeably more clean sounding.  Otherwise, there will be NO MF or LF differences in sound.   The bass will be identical -- the discrete steps available appear to allow increasing the bass way too much or make the result a little more thin.   If I can figure out the best way for a slight increase, and/or find the canonically correct bass EQ, I'll do it.   THIS RELEASE SHOULD BE CONSIDERED VERY VERY MINOR.

 

When will it be available?  Probably some-time tomorrow.   I expect that I do have the release version under test right now.

 

Link to comment

I aplogize for 2L...   I now realize the excess HF (that was a totally stupid mistake where testing code was left enabled), and the excess LF was a judgement problem.

Just tried final test (after full decodes) of 3G and it slightly misses the mark.   I make REALLY bad decisions late in the day, but now it is early in the morning.

 

3G has really clean and non-excess HF, but I violated a promise and 'fixed' the LF.  Just did not 'fix' it far enough.   It is a bit muddy, and the result of pushing up the level beyond where my original judgement said was excess.  Should have stuck with my original idea about the correct EQ sequence.  The LF sounded inadequate, so I pushed it up a little strong.

 

The bugged V4.8.3G is being uploaded as an example of one of three mistaken release candidates since the V4.8.2L.   It will take one more release attempt today, perhaps early afternoon (early evening UK/EU time) before I can test/upload it.

 

Bugged attempt: (shouldn't be too bad if you like a little excess bass.)  (perhaps approx +1.5dB too much bass around 30-50Hz.)

Bugged attempt correction coming sometime today...  The change is simple, just takes lots of time to complete.

 

https://www.dropbox.com/sh/i6jccfopoi93s05/AAAZYvdR5co3-d1OM7v0BxWja?dl=0

Link to comment

V4.8.3H should make everyone happy now...  There were some lingering BASS and HF problems, but much more minor than before.  THESE ARE FIXED.

After the pointers to the demos and decoder, there are some comments about the decoder, esp about the upcoming perfomance challenges.

 

Demos:

https://www.dropbox.com/sh/i6jccfopoi93s05/AAAZYvdR5co3-d1OM7v0BxWja?dl=0

 

Decoder (V4.8.3H is what you want):

https://www.dropbox.com/sh/5xtemxz5a4j6r38/AADlJJezI9EzZPNgvTNtcR8ra?dl=0

 

Caveat on this release:

The HF is pretty good, but the transients are a bit 'chunky' (a little strong when the transients should otherwise be 'fine'.)  This is easy to fix, but not worth making a release for it.  This HF transient matter is a 3rd order priority, barely noticeable.

 

About the decoder fixes, and later on, some 'internals' about the performance matters:

1)  Previous release had bass problems.   Went back to the beginning and reconsidered everything.  Some previous
releases had a bump in the 20Hz-40Hz region, and that is now corrected.  Also, the timing in the bass is
critical to make sure that the temporal behavior of the higher frequencies is correct.   This all seems
to be good now, including, importantly, more flat response.  The LF bump was an 'end case' situation that is difficult to determine.

2)  HF had a few problems, one is that I had left some test code enabled, and that is now fixed.   The HF was
tamed, but still seemed a little rough.   That roughness would also be manifest when using a true DolbyA based decoder,
but since DolbyA units have something called 'fog', it wouldn't have otherwise been audible.   To get rid of
the HF roughness, the input Hilbert detectors were reconsidered, and lowered the frequency where IMD is mitigated.
This Hilbert transform upgrade cleaned up the HF quite markedly.

Along with the excess bass and the HF timing, another problem is easy to cause:  stereo image damage.  This damage
is caused by bad LF and HF delays, and tends to bury vocals into the bass, therefore making the sound of vocals 'weak'. 
The EQ chains have been carefully set so that both the LF rolloff and the stereo image are maintained.


Speed-up issues:

Additionally:  did some 'low hanging fruit' efficiency improvements and reviews. I found a single thread architectural
bottleneck that will require a slight reordering/restructuring.   However, in V4.8.3H, did some low-hanging-fruit improvements
that can show a little speed up in certain cases.   An upcoming fix will move one of the time consuming operations
into another thread, therefore speeding up that slow thread to match the other, relatively slow threads.   Then,
the most impactful change wil be changing the ordering in the anti-MD code so that there is much better cache locality.

No matter what, the so-called 'sample' used in the audio processing is much more complex than one might suspect.
A 'normal'/'sane' person might think it to be an FP or DP pair of Left and Right.   The simple L/R pair is just not
enough.  Since there are four (4) bands per channel, then there would be 8 total samples minimum per audio
sample (LF-left, LF-right, MF-left, MF-right, HF0-left, HF0-right, HF1-left, HF1-right.).   Since the decoder takes
advantage of splitting the 'busy' bands into smaller bands yet, the HF0 is split into three (3) total bands, and
the HF1 is split into two (2) total bands.   If one adds it all up, there are 7 bands per channel, and two channels
for a total of 14 double precision digital samples per audio sample.   Moving those samples around is very expensive.  Even if not
moving them, referencing the 14 double precision samples still requires cache.

When thinking about 14 double precision 'sub samples' per audio sample, of course, the size of the data needs be considered.
For a double precision variable, it takes 8 bytes, or a total of 112 bytes per audio sample.   On top of that,
because of the complex layering in the decoder, emulation of 7 or 8 DolbyA decoders, and complex anti-MD code,
there are about 90 threads where the audio samples need to be passed around.   Since thread context switching
time isn't instantaneous, the data needs to be processed in 'blocks'.   I found that the optimum block size is 1536 full,
14 double precision sub-samples.

Since all of the CPU cores should be kept busy, there has to be enough data blocks to keep the CPU cores busy when 
passing the data around and doing processing.  So, after some experimentation, on my 10core machine, it appears that
approx 30 data blocks containing the 1536 samples of 112 bytes apiece is enough to keep the system busy.   For a
larger, 64 core machine, it might take 90 data blocks.   The big problem is that there is currently ONE thread that limits
the amount of concurrency in the system.   Once I fix the latency problem on the one thread, there should be good
scalability up to perhaps 32 cores instead of a little better than 10 cores.

This might give an idea of the complexity of the software, and why it is likely VERY TRUE that the decoder is the
most complex piece of software in the audio hobby set-up...

 

Link to comment

One thing about V4.8.3H that I had maybe mentioned before.  The transients for the highs might be a little stark.   That is, they might seem to be overenhanced.  (They aren't actually overenhanced, but seem so.)   Let me know whether or not another release simply to correct that problem would be helpful.   It is a very minor issue, but could be important to some people.

 

 

Link to comment

Just got some useful feedback, will cut another release ASAP.   One line of code to add, and another group of 6 lines to re-enable.

Doing official build and demos right now.

There are also some slight improvements on the HF attack shape -- more natural percussion.  (The HF attack change was already coming some time in the future.)

 

John

 

Link to comment

I uploaded the most recent try, but once the highs got under control, and my hearing working -- ended up with a smushed sound.  I know how to fix it immediately, just need to find the correct parameters to decrease/increase (change the input/output pre/de-emphasis.)   This will also allow me to extend the EQ to where I thought it should be to 24kHz or 30kHz.  (Imagine the frustration!!!)

 

Expect tonight, hopefully early in the afternoon for the release. (I hope.)

 

Link to comment

Heads up -- I haven't disappeared...

So many things are being improved that I cannot do any checkpoint or status releases right now.

 

The sound is SO clean now that the benefits of the anti-MD are so pronounced to be amazing (I mean, not amazing in a sense of improving on previous decoders -- I mean, REALLY CLEAN now.)   Best way to describe the upcoming version: very natural sounding -- I am trying to be very careful to follow good design, trusting my engineering know-how instead of being guided into oblivion by my hearing!!!

 

Basically, if you don't read the below -- the sound is TOTALLY different and extremely clean without being 'enhanced' sounding.   I am visualizing a clean 2d + depth field in the stereo image.   Before, I was hearing a good left/right with some depth.   The detail is much better -- didn't think that it would ever sound THIS good.

 

Gonna try for a release tonight, but I am finding all kinds of improvements that go far beyond and much deeper than I thought was needed.   I am seeing more and more indicators that the FA recordings being sent to consumers were NEVER intended to be decoded...   Until 10yrs ago, it would have been IMPOSSIBLE to decode an FA signal back to near -original.

 

1) found more aspects of the bass/midrange phase spread.   Getting crazy good stereo image, but being careful about the bass/midrange response, so need more testing before even a status release.   I don't want any more embarrassing holes in the upper bass/lower midrange...   There is a 'join' at about 250Hz that needs to be seamless, but there are few tells to find out if the response is correct -- just taste right now.  This makes it REALLY TRICKY to find the correct 'join' between those regions of the processing.

 

2) There was a fatal bug in almost all recent releases, actually scrambled the HF phase in some cases.  Some processing controls ran off the end of an control list, and started processing random garbage.  Frankly, I am surprised that the decoder didn't actually crash.

 

3) found cause and solution for the rough sounding vocals.  previously where the sibilance would spread/shift are much much better mitigated without anti-sibilance per-se.

 

One of the reasons for the overly intense HF (other than my hearing) was to sound more like the 'FA' recordings on the high end.   Alas, it is impossible to emulate the compressed/grainy FA sound when trying to create a near original recording.   The next results will NOT burn your hearing with excessive high end, but also don't expect the grain/false  detail of the FA original.  The decoded sound is VERY VERY clean now.

 

 

Link to comment

V4.9.0A is available.

The significant change in version number does indicate LOTS of improvements.

VERY clean sounding...  All frequencies are there, and even a slight rolloff of -0.75dB at 18kHz is very noticeable by my hearing.   Someone with really good hearing should be able to detect a very clean sound.   NOTHING is missing.   Distortion is so minimal that there is a very noticeable improvement from '--xpp' to '--xppp'.  The improvement between anti-MD modes is a noticable sense of transparency.

 

I didn't stop fixing things until everything was maximally 'good'.   The excess bass is the last real 'loose string', but isn't horrible as-is.

 

Demos (fun to listen to): -- remember that the online player doesn't do justice to the recordings.

https://www.dropbox.com/sh/i6jccfopoi93s05/AAAZYvdR5co3-d1OM7v0BxWja?dl=0

 

Decoder (better quality, a little faster);

https://www.dropbox.com/sh/5xtemxz5a4j6r38/AADlJJezI9EzZPNgvTNtcR8ra?dl=0

 

More comments:

Comments on the V4.9.0A version.

The significant change in version DOES reflect a lot of bug fixes and several types of serious improvements.

 

Lots of little things corrected, and other things verified by re-creating many of the previous EQ.

It truly is getting much closer to correct. Previous versions have been afflicted by my dislike of lots of bass, but this version has no such affliction. There were several, more profound bugs – fully fixed, and a severe fatal internal bug was amazingly not triggered. The clarity is pretty good.

 

LISTEN TO SOME OF THE DEMOS – please excuse possibly excessive bass, and on some of the recordings I did have to do some rolloff. I’ll try to come up with a more generic result after I am available again.

 

1) General decoding… A full decode uses ‘--f8’, but doesn’t necessarily need to be used. When running in ‘--f8’ mode, you might need to slightly adjust the ‘--coff=4’ value. If you use ‘--f7’, the sound will be just a little ragged. When you get that ‘ragged’ effect when using ‘--f7’, then add the ‘--hfrl=1’ or ‘--hfrl=2 switch’. Normally, most people won’t notice the slight ragged sound, maybe just use ‘--f8’?

 

2) Lots more, closer-to-correct bass. Should be close to correct, but the problem is that the EXACT rolloff frequencies need to be used. If the exactly correct LF EQ rolloff isn’t used, then a lot of recordings will need some additional correction. I used generally LESS LF EQ than I think is needed, but as mentioned above, very close to correct. If there is a problem, there will be too much low, resonant sounding bass. Other recordings will sound just fine. My caveat is not meant to imply that the bass is far off, but just the opposite – just needs some minor experimentation to complete the job. In fact, it should be damned close to correct.

 

3) My guess is that the decoder is extracting ALL available HF detail. Even using a -0.75dB rolloff at 18kHz is quite noticeable to me, and my hearing isn’t all that good. Most likely, this is giving EVERYTHING on the recording.

 

4) For anti-MD modes, to get that exquisite detail, you’ll want to use a minimum of ‘--xp’ mode, but I can easily tell the difference between even ‘--xpp’ and ‘--xppp’ modes.

 

5) The decoder should be noticeably quicker when using either high quality or raw, Dolby A style FA decodes.

 

After today, I’ll be ‘disappeared’ for several days. This will give plenty of time to play.  I have until EOD today to conveniently respond to comments (public or private), but might be able to check and respond to messages, just without access to notes, SW, etc.

 

 

Link to comment

Foolish me -- found a good solution of the bass.   Premature.

There will be a new V4.9.0B in about +6Hrs.   The only change will be a minor correction to the bass with profound positive effect.

I am running it (V4.9.0B) right now, and is sounding pretty good.

 

Have to rerun the demos, and that takes about 2Hrs, perhaps 3Hrs because of other things that are needed.

 

 

 

Link to comment

The new version, V4.9.0B with corrected bass is available.  The bass-heavy V4.9.0A is still available, but really is irritating after a while.   The extreme cleanness of the sound is shown much better by the new V4.9.0B, but perhaps just a little too much bass decrease.  As subjectively commented below, while V4.9.0A might 'feel' like 50% too much of the super low bass, the new release might be 10% too little.   The 'normal' middle bass and upper middle are much the same (e.g.40Hz to 200Hz).   The super-low bass in the 15Hz to 40Hz range are much better controlled in V4.9.0B, but not cut-off like the earlier, pre-V4.9.0 releases.

 

*My availability runs out in about 2-3Hrs, so trying to get this V4.9.0B release 'stutter' out and avaiable.   If I do anything in the next few days, it will be remotedly reading messages or perhaps listening to existing new personal decodes.

 

*If the HF is a little to 'bright' for you, your speakers, or even if there is a bit too much 'bright' in the decode results, use '--hfrl=1' or '--hfrl=2'.   These do a very slight trim of the HF.   Using EQ seems to ruin the 'fun' for me when using the '--f8' switch, because with my system combination, the clarity is 'just right'.   WIth '--f7' you might need the '--hfrl=1' or '--hfrl=2' simply to clean up the sound.   The kind of 'bright' in this decoder version is 'clean'/'crisp', shouldn't be 'slimey'.

 

The highs on V4.9.0 series are amazing and dead-on accurate so precisely that decrease in quality is significant even with very slight HF EQ.   Using --f8 instead of --fa (or --f7) is suggested.   The highs might be a little ragged at -f7, and that is because of left-over compression.  The 'ragged' sound can be partially mitigated by using the very slight EQ of '--hfrl=1' or '--hfrl=2'.   (--hfrl means 'HF rolloff'.)   When using '--f8', the sound is crystal clear -- you can actually hear the waveform envelopes become coherent when comparing  '--f8' instead of '--f7'.   There might still be some slow tilt (not damaging group delay much), but sharp response changes are well controlled.

 

Demos location:

https://www.dropbox.com/sh/i6jccfopoi93s05/AAAZYvdR5co3-d1OM7v0BxWja?dl=0

 

(Best Carpenters' results ever -- figured out their trick, but also older decoders weren't good at certain kinds of recordings.   Also, great Dire Straits.   General ABBA results are also very good -- the albums are great, but the examples are just 'okay'.)   Super good, pro quality decodes sometimes require some TLC.

 

Decoder location:

https://www.dropbox.com/sh/5xtemxz5a4j6r38/AADlJJezI9EzZPNgvTNtcR8ra?dl=0

 

Bass is kind of hard for me to deal with, even though there are a set of rules that guide the EQ to provide proper bass.  At this point, the cut-off is more of a problem than anything else.  Super highs have also given troubles, but for totally different reasons.   Bass is a problem because of my listening perception, while HF is a problem because of my bad hearing.  Frustratingly, using each kind of headphone seems to direct the EQ differently.   However, I have found that the DT990 are *really* good for HF, and are very smooth but slightly thin bass.   The DT770 just didn't have the precision that I needed in either HF or LF.  Both of my  DT770/80 might be gray market -- might be defective, but the DT990 aren't defective in ANY WAY and really helped start focusing on the best choice.   The correct listening/subjective measuring device along with respecting my own limitations -- this made getting good results much easier.  Of course, there are some people who have given good, general guidance in the correct direction.   The most frustrating thing has been the totally confusing direction also -- I don't know if it was communcations problems or even ill-intent.   However, the project now has some help that gives reliable feedback.

 

The Bass EQ scheme for frequency choice is kind of cool.   For example, when I wanted to decrease the bass, I didn't do the obvious of adding or increasing the amount of rolloff with 20Hz (or 25Hz/whatever) EQ.   Instead, it needs to be looked at as rolloff of subharmonics of 50Hz and 75Hz.   Adding 20Hz (or 25Hz) and getting the right amplitude makes the sound 'boomy'.  Since I was having troubles with the EQ (hearing burn-out after marathon rework session), I had mistakenly used a 20Hz scheme -- big no-no.

 

The best way to do the rolloff is a combination of 50Hz & 75Hz subharmonics -- and if that doesn't work, then start at a lower freq if needed.  Also, instead of using the same #dB on each subharmonic step, it is reasonable to increase the attenuation on each step.   An ideal choice (but not really needed) is if the gain at XXHz is 1.0, then the gain at XX/3Hz can be 1/3, XX/5Hz gain=1/5, etc.   This produces good time-domain behavior, but the 'flat' EQ also works, thereby producing a smoother time domain response.

 

By removing any of the vestiges of the 20Hz attempt (creating 'boom' in the sound), 50/3Hz, 50/5Hz, 50/7Hz and 75/3Hz, 75/5Hz and 75/7Hz EQs were added.  This use of odd harmonics tends to focus the stereo image and also mitigatesd boominess.   I used three sets of subharmonics, and just might need to remove the 1/7th term to add back in a bit of bass.   Instead of being perhaps 50% too much bass in V4.9.0A, now it 'feels' like perhaps 10% too little.   A little more decision is needed.   The 'rules' include filling in the EQ sequence and generally using the same, or larger amounts of EQ as the fundamental.  SO, if 50Hz fundamental is using -6dB, then the 50Hz/3 should use at least -6dB of EQ.   When using more, if needed, it is best to never use higher than -6dB in a block.  In fact, the fundamental should probably be -3dB or -6dB, and  keep everything based on -3db or -6dB.

 

There are also some 2nd order Q=0.8409 (maximally flat) EQ at 10Hz and a smooth Q=0.7071 at 14Hz.   This means that 20Hz will be affected somewhat by 2nd order EQ, but most of the 'long' rolloffs are the 1st order, and the 2nd order is intended to 'chop off' below 20Hz while the combination tends to produce weak ringing (esp with 1st order in the mix.)

 

The BASS EQ technique isn't primarly done from the 1st order standpoint, but uses a matrix scheme.   I tried a redo of the LF EQ, but based purely on step-by-step 1st order EQ.  The resulting sound had exactly the character complained about for a year or so.  The ONLY practical way of doing the EQ is with the matrix scheme, which I WILL document.  It works very well.

 

The HF EQ has been a multi-dimensional challenge until a day or so ago.   The multi-dimensional aspect of LF had been resolved for a few months, just getting the correct final EQ numbers has been tricky.   The dimensions in the HF also have to do with various frequency vs curve-shape things.   The HF results ended up being simple, but finding the 'correct' simple choice was a challenge :-).

 

Have fun...  Maybe, this time, you just might actually have fun on this release!!!

John

 

Link to comment

I purposefully disappeared for a few days to stay away from the decoder so I could judge with virgin hearing.

 

Some of the demos were a bit thin on the LF, ALL of the thinness was the EQ choice for rolling of excess LF.

 

The latest sounds BETTER than I thought, and the HF has the 'resonance' of being correct.   A slight rolloff might be in order for casual listening.  Using --hfrl=1 on some recordings can be beneficial (that is a sub-1dB rolloff at the highest frequencies starting at 18kHz (-0.375dB) or so.  Even the very slight change of -0.15dB at 12kHz can really help.

 

New demos, modified EQ for some of the exceptions, with exactly the same decoder (V4.9.0B) will be produced for tomorrow (-1 release instead of -0).

 

PS: have truly perfect ABBA decodes waiting in the wings.

 

Link to comment

ANNOUNCEMENT:  V4.9.0B is going to be the last major quality improvement release for a while.

Feedback is that the quality is VERY good, still maybe a few rough edges.   Any real changes needed are at the +-1dB level at most.   Much of that would be differences in master tapes and diddling done before FA encoding.

 

* Next major release, est: 1-2wks will have the decoding speed improvements, perhaps some more minor quality improvements as more feedback comes in.

 

The first set of demos (second set) had a few errors in the recordings that need exceptional EQ (mostly too thin) and so the decodes are all being redone.

 

A request was made by one of the testers to freeze the decoder for a while.   The quality is REALLY very good, and the HF is very clean, but sometimes needing a slight tilt downwards. (-0.375dB at 18kHz is about all that is needed.)   Other small changes might be needed in the future, but as is, is pretty damned good.

 

I plan that the next major release (V5.0.0) will have the planned speed ups.  I think that I have chosen the correct method, which has least liklihood of breaking things, and the best chance at a 2-3X speedup at the highest quality levels.   I believe that the lowest/lower quality levels should be showing a noticeable speedup also.

Link to comment

As a small portion of my commemoration of everything going on with ABBA -- here are snippets from the ABBA/ABBA album, done in the highest quality possible in the decoder (--xpppp=max) and with reasonably careful post-decoding EQ (sum total of -0.90dB in mostly -0.1875dB steps from 9kHz, 12kHz, 15kHz, 18kHz, with an extra -0.1875 at 12kHz -- there was a rational reason for the extra 12kHz EQ.)   So, given the kind of EQ being done, the effective amount of EQ at 9kHz is around -0.1dB, and hits around -0.60dB at 18kHz.  No special bass EQ.  Versions of the decodes for this album are in the usual places, here are the snippets:

 

https://www.dropbox.com/sh/q1u8vmx0xktqs3l/AABw5gGSCmsxV5GpgDxwX_-7a?dl=0

 

All 8 of the standard albums are decoded to this quality level, but the post-EQ and normalizing the levels haven't been done on any other albums yet.

 

Link to comment

Important update:

 

After a lot of additional testing, I believe that the bass is a bit too thin, and I have a temporary correction that can be made without a new release.   The new demos will have this correction, and I suggest that until the next release comes out, ALL decodes use the following addon to the list of switches:

 

--pvdl=100,3

 

There is a LF rolloff between 100Hz and 250Hz that is easy to correct in the decoder, but for now, I am not going to do a new release.  Simply add the command above to the command list, and the result will be very similar to the next release.   WHen I do the release, I'll determine the correct frequency -- 100Hz or 250Hz (it will be one of the two.)

 

The ABBA demos will also be updated.

Link to comment

QUESTION TO EVERYONE:

Is my hearing telling me lies about the level of the bass?   I'd need & greatly appreciate some feedback on this.

 

UNTIL THERE IS FEEDBACK TO THE CONTRARY -- I am assuming that the EQ as of V4.9.0B is CORRECT!!!

 

It is plausible that the bass below approx 125Hz is 3dB too light. (actually about -1.5dB at 100Hz, about -2.25dB at 50Hz, -2.75dB at 20Hz.)   It is also plausible that the bass EQ Is correct.  There is a boundry between -3dB sequence of EQ and -6dB sequence of EQ.  This is kind of complicated, but if the value should be -3dB, then the frequency of the EQ should be 125Hz.   If the value should be -6dB (as it is), then the frequency of the EQ should be 100Hz.  The specific frequency is not as important as if the bass is weak or not.   I know that this kind of error seems insignificant because most transducers are NOT accurate very far below 100Hz (even though they purport to be), and even accurate room EQ Is tricky to do.

 

This matter of the weak bass is technically plausible either way -- depending on what the original encoder design is.   I don't want to infinitely embasass myself further with weak bass, but the frustrating matter is that with my hearing, the EQ somemtimes sounds good, and sometimes slightly light LF.   This really is NOT a matter of taste, but is definitely a matter of correctness.   Finding correctness is exceedingly tricky difficult without specification and with unreliable and poor hearing.  Help with decision making, EVEN IF JUST OPINION will point me in the right direction.

 

I understand that for now, the EQ is probably close enough -- the quality of the HF is truly a breakthrough.   However, the decoder MUST be as accurate as we can all make it.   The internal fix is already planned -- I just don't know whether or not to do it.

 

The updated demos will be held back until we know the correct answer, further demos based on V4.9.0B will be produced unless change is needed.  I don't want to stutter more crazy releases!!!   This only confuses everyone.


THANKS!!!
 

Link to comment

Status on decoding the standard, unadulterated (Polydor/Polar/original Atlantic/etc) ABBA CDs is very good.

The decoding is totally out-of-the box, just follow normal decoding rules, and no, NONE, zilch, nada changes to calibration, stereo image, etc.

 

The only 'tweaks' for decoding are the below EQ:

--pvdh=15k,-0.1875 --pvdh=9k,-0.1875 --pvdh=12k,-0.1875 --pvdl=75,-3 --pvdl=25,6

(Some versions might benefit from using -0.375 instead of -0.1875 in the HF EQ settings above.)

 

Above EQ is the 'average' for all of the recordings, where the 'ABBA' album itself might need a little more HF rolloff (perhaps -0.375 at 12k instead of -0.1875.)   These very slight EQ settings are important -- the decoding is *THAT* precise.   It is amazing, considering the extreme amount of processing going on.   Even the self titled 'SuperTrouper' cut on the 'SuperTrouper' album is coming out very, very good.   THAT was one of the windmills that I had been tilting for the last 5yrs.

 

The bass EQ is a little troublesome, and might result from an EQ bug -- not sure.   However, suffice to say, on the ABBA albums the -3dB at 75Hz and +6dB at 25Hz really does improve the results.

 

So, to decode an average, unadulterated ABBA album (none of the 'Complete Studio Recordings' nonsense), the following command should give good results:

 

da-avx --info=1 --infile=inCD.wav --overwrite --outfile=outCD.wav --f8 --xpp --pvdh=9k,-0.1875 --pvdh=12k,-0.1875 --pvdh=15k,-0.1875 --pvdl=75,-3 --pvdl=25,-6

(using --xppp is better than --xpp, depending if you have a fast enough computer.)   If you have something like a 2 core Atom, then given --fx or --fz a try, it does run much faster, but produces a slightly muffled sound.   All of my own ABBA demos are done with --xpppp=max, which takes FOREVER even on my 10core X CPU.   WIth an 18 or more core machine, the decodes even at --xpppp=max should be almost tolerable to wait for.

 

The '--pvd' EQ stuff isn't really 'necessary', but does provide a noticeable improvement.   Translating the 'FA/DA decoder' EQ into technical speak, it is the following:

 

1st order HF shelf, 9kHz, -0.1875dB

1st order HF shelf, 12kHz, -0.1875dB

1st order HF shelf, 15kHz, -0.1875dB

1st order LF shelf, 75Hz, -3dB

1st order LF shelf, 25Hz, +6dB

 

 

Link to comment

After a LOT of verification and decoding of various materials, I am fairly confident that the LF EQ on V4.9.0B is correct.

 

In my previous messages, I had mistakenly used the term 'matrix EQ', and the more proper term is 'lattice EQ'.   It is a very interesting scheme which matches the encoding EQ characteristics, creates a very smooth sound, while still providing the desired rolloff (or boost.)  There are 4 parameters to the lattice EQ:   EQ freq, EQ gain, delta freq A, and delta freq B.   The key to the very smooth sound are the 'delta freq' portions of the EQ.   For example, for the 250Hz range, it appears that a good value for delta freq A is about 150Hz and delta freq B is about 20Hz.  Of course, these are variables and the correct value depend on whether or not EQ is 'de-emphasis' or just used for listen ability reasons.

 

One reason whey I had so many troubles with the LF EQ is because EQ like normal 1st order -6dB or -3dB at a given freq simply does not produce correct results.  Certainly 2nd order or higher order EQ didn't work correctly either.

 

Instead of a normal 'EQ' scheme, it took about 1-2yrs to recognize that there was a more complex 'regularized' scheme used for EQ throughout the FA encoding scheme, and most of the EQ is a derivative of this lattice EQ.,   Earlier, I had detected a simpler scheme before the correct 'lattice' was eventually uncovered, but even the first step upgrade was not correct. Once the lattice scheme is determined, there is, of course, the 'frequency' and 'rolloff or boost' specification, but there are two other dimensions to that specific EQ (basically four dimensions.)   These parameters of the EQ also interact with other EQ mechanisms...   So, this is where the 'Tetris' game starts getting played.   The 'zeroing in' on the design first starts being whack-a-mole and rabbit chasing.   Once the design concept for the specific EQ application is understood, then the rabbit chasing starts improving to a game of Tetris.

 

The game of 'Tetris' starts manifesting when there are several paths of EQ, many lattices, and there is a challenge where the EQ can be applied before and/or after a nonlinear gain control element.  So, the EQ doesn't simply accumulate, but has subtle interactions where incorrect settings might either create a frequency response imbalance or a dynamics problem where (as you have heard on earlier decoders) overly intense or suppressed dynamics.   The subjective effort starts changing again, from a game of Tetris to successive approximation or perhaps, in the best case, bisection.,.

 

Most of the time in these last couple of years has been to figure out the general architecture for the EQ.   There are very many possible 'close approximations', but these approximations will break down and produce eccentric results.   It took numerous of these architectural test cases, to eventually be able to do successive approximations, starting to converge on a most accurate result.  *I am very willing to bet that any UN-adulterated FA materials can cause the decoder to produce eccentric results, as long as the decoder is used correctly.

 

After coming up with many plausible solutions to this complex problem, many ended up being reasonably good.   Some 'solutions' had defects like response balance issues, a week or so ago, with the lattice EQ, it starting being apparent that the true solution was appearing.  With my poor hearing judgment, I was certainly fooled into thinking that a choice sounded good, but did not.   These choices are NOT tweaks, but are building blocks...   If the choices were 'tweaks', the decoder could NEVER be brought up to a level as good as it is today.,

 

This true solution not only has plausible dynamics behavior, it truly has accurate dynamics behavior.   The only worries at this point are at the +-1dB error type thing at the highs and lows.   1dB error can still be painful, but SOMETIMES deviations away from the standard decoder solution are needed.   Generically, I am finding that deviations of -0.2 to -0.4dB at 6kHz are desirable.  The same can be helpful at 9kHz, 12kHz, further 3kHz increments all the way up to 24kHz or higher.   Many recent recordings need NO deviations, some older recordings are happiest with -0.1875, -0.375 deviations, and sometimes as much as -0.75dB, but not often.

 

Why the strange 0.1875, 0.375, 0.75, 1.5dB increments?   They appear to be standard usage, and I have seen up to 6dB.    I'll be doing some future experiments on the decoder where I intend to try -9dB in one set of EQ.  Why try -9dB?   Because there is a series of several EQs in a certain chain, and want to make sure that the multiple EQ cannot be collapsed.   One important thing about EQ -- a series combo of -3dB and -6dB are NOT the same as -9dB.   In fact, sometimes using -3dB, -3dB and +6dB at the same frequency in tandem can be useful in certain situations,.

 

So much for the tech lesson, but wanted to explain on small part of the complexity and tedium of reverse engineering the decoder.   I am still not 100% that all of the little numbers are correct, and in the background chasing possible alternative choices, perhaps attaining higher quality.  When users give cogent and meaningful feedback, it will definitely be considered for creating improvements.

 

John

 

Link to comment

Currently working on the improvement/update release starting with V4.9.0B.

The very much improved version will be available ASAP, but not before it is ready. (a day or two.)

 

There are going to be several improvements, two most important ones:  bring the bass up to more realistic levels and -- just figured out a missing HF EQ.   The new add-on improvement is 'yet another phase scrambler' found and a solution for correcting the 'scramble'.

 

Fix 1:  bring the bass up to a more plausible level while considering the various ramifications of the change.

Fix 2:  Add-in missing 9kHz EQ along with removing work-arounds.   This missing 9kHz EQ was  a clear *mistake* and not uncovering a new kind of processing.

Fix 3:  Just uncovered the precise method of HF phase scrambling.  There has been some evidence that there is HF phase scrambling in the audio, now the evidence has been accumulated, studied and a solution created.

 

1)  The bass issue.  Fixing the bass is tricky, and can be confusing.   The bass EQ does lots of timing shifts simply because of the effects of timing by the LF and MF EQ.   This creates a 'keyhole' through which the higher frequencies phase matches the MF.  When changing the bass EQ, then the HF/MF 'keyhole' needs to be considered.   Essentially, this makes the bass EQ have 'steps' that are more complex than just adding/subtracting 3dB or 6dB at one frequency or another.   If EQ is just thrown at the signal as a blob, then the apparent HF can suffer in either severe or subtle way.   This HF/Bass keyhole thing is also one of the 'tells' that I use so that once the EQ is in the correct region, then the keyhole can be used to zero-in on the nearest 'correct' bass EQ.   Sometimes, I choose the 'wrong' EQ to start with...   This time, the base choice for EQ was 2-5dB too little bass (depending on frequency.)   The previous (V4.9.0B) bass EQ resulted from choosing the wrong EQ base value.

 

2)  The missing EQ step on HF.   I am VERY dependent on technical correctness as my hearing is not very reliable.  Silly me -- I missed a 9kHz EQ step -- I just didn't/couldn't strongly notice it as an error.   This error is corrected, plus a few associated changes were needed.   The need for this bugfix comes from nothing but a *stupid mistake* and NOT a technical challenge.  This mistake was noticed because of the revelation associated with step '3' below.

 

3) Newly found missing de-scrambling step.  The result of this correction/update is NOT profound but is a qualitative improvement.   Adding in the missing phase descrambling changes the sound from 'plausible' to ' noticeably nicer'.   This is the kind of improvement that comes from a notice that 'sounds okay but something is wrong' to 'significantly better'.   I believe that this scrambling is intended to keep the signal from developing anomalies and instead glosses over strange things that might happen.

 

There are test-demos being run right now.  Often, when doing test runs, problems/bugs are sometimes found, needing resolution and restarting the test demos again from scratch.

Was expecting to send the release a few hours ago, but it couldn't happen.  The new release, which REALLY sounds better yet, will be made available ASAP.  It might be today, tomorrow, not likely delayed beyond Thursday.

 

 

Link to comment

V4.9.2B is now available.

Demos:

https://www.dropbox.com/sh/i6jccfopoi93s05/AAAZYvdR5co3-d1OM7v0BxWja?dl=0

Decoder:

https://www.dropbox.com/sh/5xtemxz5a4j6r38/AADlJJezI9EzZPNgvTNtcR8ra?dl=0

 

The improvements are mostly subtle adjustments giving qualitative much better results.

1) Highs more clean

2) Bass more pronounced/more accurate.

3) Stereo image much, much better.

 

This has actually been available for just under 1day, but has been going through final testing/review.  The improvements during transition to V4.9.2B are most likely a more profound improvement than the transition to V4.9.0B.  Audibly, there is a sense of clean, distortion free sound.

 

I want to clarify abou the 'stereo image' changes.   I mention this on many of the previous releases.   The stereo image is NOT being changed by the conventional manipulation of the Left/Right or Mid/SIde channels together.   The 'stereo image' changes/improvements are made by making the time delays vs. frequency on each of the channels more accurate, therefore more accurate aural time domain representation.   There are NO quad-type games being played, even though it might seem like it when listening to the more profound improvements.

 

The bass improvements are very tricky because there is a 'keyhole' effect for the MF/LF time delay relationships.  When using 1st order EQ, the filters can affect frequencies up to almost a decade away.  This effect isn't just frequency response, but also includes phase.   This is where the 'keyhole' effect occurs, and unless the time delays are very carefully considered, there can be an audible loss of HF, even though they are actually intact.   If the time delays aren't very accurate, the lows/MF can obscure the highs, so I have a lot of time has been spent making the bass more accurate without an apparent (but not really real) loss of HF.

 

The more clean highs result partially from the tedious handling of the pre/de-emphasis.  That wouldn't be terribly difficult except there is also another two layers of EQ on top of the pre/de-emphasis.   Even though previous versions got the pre/de-emphasis correct from a static standpoint, I believe that this version now has very good gain tracking over the entire audio band.   Static tracking can be tricky, tracking so that it seems correct over part of the HF and LF is more tricky.   The very tedious testing, while also the decoder NOT being flat (nor should it be) has been quite a challenge.   There ARE portions of the decoder that should be flat, and it indeed IS flat over those internal portions, but the input/output can-not be flat and sound correct.

 

* Trying to measure the decoder like an amplifier or pre-amp (even RIAA) is bordering on folly.  It is *interesting* to do such measurements, but of little use when measuring the process as a whole.   There are so many time domain aspects of the decoder that the only way to guarantee perfection is starting with the specification.   I believe that after numerous iterations and chasing rabbits -- the decoder is pretty darned accurate.  

* Many recordings have had 'sweetening' done to them.  Usually, the EQ can be counteracted by a little bit of 1st order EQ.   The before-encoding EQ isn't usually very strong.

 

 

 

 

Link to comment
1 hour ago, John Dyson said:

V4.9.2B is now available.

Demos:

https://www.dropbox.com/sh/i6jccfopoi93s05/AAAZYvdR5co3-d1OM7v0BxWja?dl=0

Decoder:

https://www.dropbox.com/sh/5xtemxz5a4j6r38/AADlJJezI9EzZPNgvTNtcR8ra?dl=0

 

The improvements are mostly subtle adjustments giving qualitative much better results.

1) Highs more clean

2) Bass more pronounced/more accurate.

3) Stereo image much, much better.

 

This has actually been available for just under 1day, but has been going through final testing/review.  The improvements during transition to V4.9.2B are most likely a more profound improvement than the transition to V4.9.0B.  Audibly, there is a sense of clean, distortion free sound.

 

I want to clarify abou the 'stereo image' changes.   I mention this on many of the previous releases.   The stereo image is NOT being changed by the conventional manipulation of the Left/Right or Mid/SIde channels together.   The 'stereo image' changes/improvements are made by making the time delays vs. frequency on each of the channels more accurate, therefore more accurate aural time domain representation.   There are NO quad-type games being played, even though it might seem like it when listening to the more profound improvements.

 

The bass improvements are very tricky because there is a 'keyhole' effect for the MF/LF time delay relationships.  When using 1st order EQ, the filters can affect frequencies up to almost a decade away.  This effect isn't just frequency response, but also includes phase.   This is where the 'keyhole' effect occurs, and unless the time delays are very carefully considered, there can be an audible loss of HF, even though they are actually intact.   If the time delays aren't very accurate, the lows/MF can obscure the highs, so I have a lot of time has been spent making the bass more accurate without an apparent (but not really real) loss of HF.

 

The more clean highs result partially from the tedious handling of the pre/de-emphasis.  That wouldn't be terribly difficult except there is also another two layers of EQ on top of the pre/de-emphasis.   Even though previous versions got the pre/de-emphasis correct from a static standpoint, I believe that this version now has very good gain tracking over the entire audio band.   Static tracking can be tricky, tracking so that it seems correct over part of the HF and LF is more tricky.   The very tedious testing, while also the decoder NOT being flat (nor should it be) has been quite a challenge.   There ARE portions of the decoder that should be flat, and it indeed IS flat over those internal portions, but the input/output can-not be flat and sound correct.

 

* Trying to measure the decoder like an amplifier or pre-amp (even RIAA) is bordering on folly.  It is *interesting* to do such measurements, but of little use when measuring the process as a whole.   There are so many time domain aspects of the decoder that the only way to guarantee perfection is starting with the specification.   I believe that after numerous iterations and chasing rabbits -- the decoder is pretty darned accurate.  

* Many recordings have had 'sweetening' done to them.  Usually, the EQ can be counteracted by a little bit of 1st order EQ.   The before-encoding EQ isn't usually very strong.

 

 

 

 

Stupid me - I didn't wait long enough for my hearing to work correctly...   I thought that I waited long enough to let the correct decision settle in -- WRONG.

V4.9.2C is coming out later today with the fix below.  WIsh that I waited another hour or so.   There are 20Hrs of decodes with V4.9.2B, and now the time is wasted.

 

There is about 1-2dB too much at 20kHz, then having audible effect down to 10kHz or so.   The problem is easily fixable, and I want/need to finish this part of the project.   The performance (decoding speed) matters are now paramount,.

 

I have some decoding formulas -- perfect ones, for Supertramps 4 albums that I use for testing.  Each album is slightly different, so it is helpful to know these setups.   I'll present the Supertramp decoding formulas after the V4.9.2C that is coming out in a few hours.

 

The only decoder change from V4.9.2B to V4.9.2C is correcting the excess 1-2dB  (that is really all the error.)  It is actually a 1st order +3dB error at 30kHz, which actually DOES apparently have a very important impact in the audible range.   In fact, some of the buggy extreme HF in earlier versions of the code came from a lack of consideration for the needed out of band EQ.

 

I left the +3dB error at 30kHz because it seemed to be unimportant -- I couldn't hear it.   Making changes can be frought with numerous troubles, so I didn't want to fix the problem given the inaudbility of the error.  An hour or so ago, I noticed a brittle HF sound, therefore corrected the +3dB error at 30kHz, and the problem appeared to go away.  Many of my errors come from inability to hear troubles, and then making the wrong prospective decision (many of my engineering predictions are correct, but eventually make the wrong decision because the result sounds wrong based on my bad hearing.)

 

When starting this project, I didn't realize the extreme accuracy requirements even maintained outside the audio band.  If the decoder had fewer variablews, maintaing the accuracy would have been simple.   Note the hoopla about RIAA EQ?   That is totally, absolutely *trivial* when compared with the requirements in the decoder.   Along with that, there are no specs.

 

Link to comment

V4.9.2P is now ready.   Sounds much better -- very careful reset of ALL settings. Hopefully much less excess intensity in the HF.  (Part of that intensity came from an attempt to emulate the false FA bright sound.)   Basically, the 0.27dB error screwed up some other EQ choices.  Many previous changes and the 'worrisome' revelation yesterday forced a review of how legacy settings had been affected by new changes through the last few months, however I previously didn't do a global review from scratch.   There are almost NO settings in the HF that weren't carefully  reviewed.  The 0.27dB bug has existed almost since day one -- I just assumed that the signal was flat at the position where the offset error was found.

 

* EXCEPT FOR THE TYPE-O FIX, the changes result from total reset and review of every little step in the HF and LF EQ.

 

The changes encompass so many things that enumerating them isn't helpful here.

 

Demos:

https://www.dropbox.com/sh/i6jccfopoi93s05/AAAZYvdR5co3-d1OM7v0BxWja?dl=0

Decoder:

https://www.dropbox.com/sh/5xtemxz5a4j6r38/AADlJJezI9EzZPNgvTNtcR8ra?dl=0

 

I regret the 'freak-out' attack, but when reviewing the fix for the the upper rolloff, yesterday I had just found another type-o bug.   This NEW type-o bug was the stronger of three reasons why the HF was too intense.   I have to assume some things because of the hearing matter.  I MUST depend on engineering decisions, and when I don't detect a type-o, it ends up being terrible wasted time.  Very often, I cannot hear the effects of a type-o!!!!

 

 

Details of yesterdays' 'panic':

 

The specifics of the 'freak-out'/panic attack came from an EQ sequence that should have been 6k/12k/18k, but instead was 9k/12k/18k.  This screw-up is easy to make because most such sequences used elsewhere DO start at 9kHz instead of 6kHz.   That error caused my heart to sink.  When starting to review that section of code that also contained the 30kHz rolloff, finding that bug, it made me distrust a lot of other things.   It is SO EASY to over look that kind of error. I couldn't hear the bug. (The error is on the order of about +1, perhaps +2 dB in the 6kHz to 9kHz freq range.)

 

Good news about the speedup coming in a week or so:   figured out how to get the same quality from about 1k FIR taps as what previously needed 7000.

 

 

 

Link to comment

Got some guidance that I screwed up the HF again.   That will be fixed today (it does sound good ignoring the HF matter), but will also have the speed improvements below.

 

 

Now focusing on speed, and also still reviewing the HF/LF balance.   I can hear that the lower parts of the HF are mixing very well with the MF (previously had a 0.27dB error), but not 100% sure about the highest frequencies as there might be an HF problem that I got fooled by again*.  Also, the LF had been tricky, and might still be a little weak, but now doesn't have the bump at the lowest audio frequencies.   Previous versions had an extreme LF (20-30Hz) bump, but hadn't figured out how to eliminate it.   (As mentioned before, the EQ isn't just throwing in a filter, but the filters are more complicated than simple 1st order (or 2nd order) EQ.)  * There is an HF EQ sequence -- architecturally suggested -- that I removed because it seemed wrong.   I am re-inserting it.   Best to keep my engineering hat on, because my hearing just sucks.,

 

This morning, got a 20% speed increase for the default quality mode -- also a slight quality improvement, also for DolbyA emulation.   This is about all of the 'low hanging fruit' in the basic engine.   The high quality modes will be decreased from about 6 steps of modes to perhaps 3 along with the 'default' mode.   The first step will be approx the same quality, maybe better than --fz quality, but the speed will be about the same as --fx.   The next step is planned to be --xpppp quality, but about the same as --xp speed.   The third step might either be nonexistent, slightly better than the previous step, or very noticeably better than the previous step, but --xpp speed.   The speed/quality increase will take about 1-2 days, but also might be one of those 'rabbit holes'.   I do believe that the planned method will produce very accurate results over a wide frequency range.  (The nonlinear gain control processing produces sidebands that need to be filtered out.  Once those sidebands are gone, the total resulting 'fog' is less than the original, normal gain control.)   The only way to distinguish the sidebands is by using 90deg phase shift/Hilbert transforms.  It should take about 1week to get these working.  (Will actually be working in about 1 day, but testing/verification takes quite a while.)

 

It is very possible that the decoder will start being more practical in a 4 core machine...   On my machine, it is now running about 4X faster than realtime.

 

 

 

 

 

Link to comment

The release later today, will have the HF EQ that is what I had engineering-wise predicted, TOTALLY IGNORING my hearing.  *  that is, ignoring the HF intensity from my hearings standpoint.   There is an EQ that seems correct from a technical prediction standpoint -- but my hearing says that it is too dull.   From the feedback that I keep getting, I keep burning peoples hearing with intensity.

It does not inspire confidence when the sensory reality doesn't agree with the engineering reality.  Maybe like flying a plane on instruments.

 

My hearing starts causing me to lose confidence in my engineering predictions -- and produce shrill stuff that *sounds good* when I send it out.  I am getting tired of this, not because of anyones frustration and rightfull loss of interest, and I'll never quit until it is working perfectly, but the ups and downs get frustrating.

 

Let me know if the sound is any better -- I am adding the standard 6k/9k/12k/18k sequence with the 24k/30k above audibility EQ.   I should have realized there was a problem when I couldn't tell the difference when the 24k/30k EQ was enabled/disabled.   The difference when that (24k/30k EQ) enabled should be about 2-3dB at 12kHz.   That is, let alone the sequence that I had shortened to 6k/9k, because it SEEMED right, but I got the heebie-jeebies when removing the others in the sequence.

 

All of the EQs are -6dB 1st order each step, except for 30k being +6dB (should be +3dB, I think, but there might be some advantages because of the IIR EQ to using +6dB.)   This EQ problem tells me that my hearing sometimes is down -6dB at 10kHz, but late this morning, I could hear it, up to 14kHz and beyond... 

 

John

 

Link to comment

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now



×
×
  • Create New...