'FeralA' decoder -- free-to-use

[John Dyson]

The decoder is becoming so good that the panic attack yesterday came from a mistakenly perceived minor problem.

It is making the raw consumer recordings really seem inferior to the decoded result.

 

Below, note the addition of the word 'BLUR' to the terminology.  It does NOT mean a loss of frequency response, and doesn't actually mean loss of stereo image, because the decoder can recover the stereo image MUCH MORE COMPLETELY NOW, remove whatever distortion sidebands exist, and also recover some transients making them more detailed.

 

I uploaded several  'easy to access' examples, showing the detail obscuring effects of the 'BLUR' in undecoded materials.   Many of Olivia Newton Johns older recordings are very noticeably improved.   By my perception, about half that I have reviewed so far are profoundly improved.

 

The BLUR is not really 'loss of detail', but is more 'hiding detail', and since the detail is not really lost, the decoder can do a couple of things to remove the 'veil' from the recording.

There are a few examples immediately in the URL, and be careful to use headphones or some other environment that maintains a clean stereo image.   It is probably impossible to hear the improvement on computer speakers, so dont' be disappointed unless a system can maintain a stereo image and/or the transient/detail behavior is very good.   The improvement is both subtle and profound.

 

https://www.dropbox.com/sh/i6jccfopoi93s05/AAAZYvdR5co3-d1OM7v0BxWja?dl=0

 

[John Dyson]

No real bugs except perhaps a slight lack of expansion in the 6kHz region and a preference for a different EQ.

 

The change/bugfix and the abililty to select a different EQ will be started after the upcoming weekend.  I don't want to risk destabilizing the decoder and would like to use the decoder myself.   Also, at least an up-to-date starting document is really needed.   SInce there is only me doing the programming, I'll be starting the change/addition right after this upcoming weekend.

 

A solution for the 6kHz expansion problem hadn't been figured out until today.    Of course, the EQ is technically triiviail, but getting the EQ just right might take a little while (hours, not more than several days.)

 

Have fun with the decoder, it is still at V7.0J, and have made no changes  to the source or even built any test versions since the previous release.   The basic/central decoder is for all intents and purposes *complete*.   Other than the minor changes mentioned above, in the future, on the order of multiple weeks, there might be improvements in speed that require some structural but not algorithmic changes.   Also, there would be some prep for the GUI later on.   For the GUI, the intent is to use an interface language, connected through a TCP stream to the GUI itself.   This method will keep from disturbing the internal structure of the decoder.

 

I am near the end of doing some decoding on ABBA that is mostly profoundly better than probably anything heard since the studio.   During the rather tedious work,  I found the EQ that you normally hear is very likely NOT the mix created while doing the recording.   Apparently the mix normally distributed was intended to change the spectrum to be a little better for broadcast.  The EQ is not the 'common sense' kind of 'shape', but is more like the internal EQ in the decoder.  The 'ultmate' sound is more clean, and more distinct stereo image.   Previously, I never got a totally satisfactory decoding result on the ABBA recordings.  The decoding results, just starting to do final QC until the weekend, should be likely be preferred when using current technology equipment.   The result will not be totally dependent on my 'taste', but the optimal EQ is based on tells associated with the response curves between bands match/don't match.    There are often several aliases when using the tells, and that is where my taste becomes involved.  None of the ABBA decoding entails any decoder changes, becuase changes for the purpose aren't needed.

 

So no changes until after the weekend.   Whatever changes for normal operation are very small.   There will also be added EQ functionality for an existing switch.

 

 

[John Dyson]

A release tomorrow is planned, perhaps 2/3 chance.   There has been a  problem with the dynamics of the trumpet on 'Dire Straits' 'Your Latest Trick'.   I was told that the error is minor, but any error like that is unacceptable.

 

After trying the prospective change, it appears to improve the results by a slight, likely correct, amount.   The change is to again fully enable the 6kHz band on the descrambler, which increases the expansion, improving the 6kHz dynamics a moderately small amount.   Once the descrambler is changed, then the freq response needs to be re-flattened.   Good news about the response, once I figured out a good way to flatten it.  Now the response measures much more flat, now the measurement is 0.02dB variance between 3kHz and 7kHz, which is really, really good.   All stages follow the '3dB' or simple submultiple rule, and actually using two straight 1.5dB EQs at standard freqs of 2.99025kHz +1.5dB 1st order and 4430Hz -1.5dB 1st order.  (yes, those are totally standard freqs which nominally represent 3kHz and 4.5kHz.)  And yes again, the precision is critical.  Perhaps the 2.99025kHz could be rounded to 2.990kHz, but no more error than that.   The max avg level of 0.34dB from 20Hz to 20kHz comes from a frustrating peak in the 150-200Hz range, and I have really tried to get rid of it.  The peak is actually about 0.4dB above the average gain around the area.

 

Frustrated with myself, but I knew how to do the response correction to the precision we have now, but somehow I forgot.  This is majorly complex stuff.

The resulting response in the upper midrange to HF is below.  The rolloff above 9kHz is not an actual rolloff.  There *Is* a dip between 1.5kHz and 3kHz, is to be expected, and probably undesirable.   There is also a small difference between the starting LF response of 0.17dB at 1kHz and the HF of 0.24dB at 3kHz, but the dip down to 0.04dB at 1.5k to 1.6kHz is probably not good.  The base LF response actually increases to about 0.24dB, then does the changes that it is supposed to, but unfortunately reaches about +0.45dB about 100Hz wide in the 200Hz range as mentioned earlier.   Both the dip in the upper MF/lower HF and the 0.45dB might be artifacts of the design, or perhaps result from rounding component values in the hardware doing the encoding.

 

To me, this is d*mned flat for something so complex, doing lots of gain control.   Frankly, I never thought that the decoder could/would ever be this flat.

 

John

 

LEVELS 20Hz to 20000Hz
dB raw: -23.56 dB dec: -23.22 dB diff: 0.34
LEVELS 3000Hz to 20000Hz
dB raw: -40.21 dB dec: -40.13 dB diff: 0.08

LEVELS 1000Hz to 1500Hz
dB raw: -42.61 dB dec: -42.51 dB diff: 0.1
LEVELS 1500Hz to 3000Hz
dB raw: -41.71 dB dec: -41.59 dB diff: 0.12
LEVELS 1000Hz to 3000Hz
dB raw: -37.81 dB dec: -37.71 dB diff: 0.1
LEVELS 1000Hz to 1100Hz
dB raw: -46.24 dB dec: -46.07 dB diff: 0.17
LEVELS 1100Hz to 1200Hz
dB raw: -46.89 dB dec: -46.77 dB diff: 0.12
LEVELS 1200Hz to 1300Hz
dB raw: -47.35 dB dec: -47.26 dB diff: 0.09
LEVELS 1300Hz to 1400Hz
dB raw: -47.66 dB dec: -47.60 dB diff: 0.06
LEVELS 1400Hz to 1500Hz
dB raw: -47.91 dB dec: -47.86 dB diff: 0.05
LEVELS 1500Hz to 1600Hz
dB raw: -48.14 dB dec: -48.10 dB diff: 0.04
LEVELS 1600Hz to 1700Hz
dB raw: -48.40 dB dec: -48.34 dB diff: 0.06
LEVELS 1700Hz to 1800Hz
dB raw: -48.68 dB dec: -48.61 dB diff: 0.07
LEVELS 1800Hz to 1900Hz
dB raw: -48.98 dB dec: -48.90 dB diff: 0.08
LEVELS 1900Hz to 2000Hz
dB raw: -49.30 dB dec: -49.19 dB diff: 0.11
LEVELS 2000Hz to 2100Hz
dB raw: -49.62 dB dec: -49.50 dB diff: 0.12
LEVELS 2100Hz to 2200Hz
dB raw: -49.95 dB dec: -49.81 dB diff: 0.14
LEVELS 2200Hz to 2300Hz
dB raw: -50.28 dB dec: -50.12 dB diff: 0.16
LEVELS 2300Hz to 2400Hz
dB raw: -50.60 dB dec: -50.42 dB diff: 0.18
LEVELS 2400Hz to 2500Hz
dB raw: -50.91 dB dec: -50.72 dB diff: 0.19
LEVELS 2500Hz to 2600Hz
dB raw: -51.21 dB dec: -51.00 dB diff: 0.21
LEVELS 2600Hz to 2700Hz
dB raw: -51.50 dB dec: -51.27 dB diff: 0.23
LEVELS 2700Hz to 2800Hz
dB raw: -51.76 dB dec: -51.53 dB diff: 0.23
LEVELS 2800Hz to 2900Hz
dB raw: -52.01 dB dec: -51.77 dB diff: 0.24
LEVELS 2900Hz to 3000Hz
dB raw: -52.25 dB dec: -52.00 dB diff: 0.25

LEVELS 3000Hz to 4000Hz
dB raw: -49.18 dB dec: -48.94 dB diff: 0.24
LEVELS 4000Hz to 5000Hz
dB raw: -50.66 dB dec: -50.45 dB diff: 0.21
LEVELS 5000Hz to 6000Hz
dB raw: -51.13 dB dec: -50.89 dB diff: 0.24
LEVELS 6000Hz to 7000Hz
dB raw: -51.56 dB dec: -51.34 dB diff: 0.22
LEVELS 7000Hz to 8000Hz
dB raw: -52.39 dB dec: -52.28 dB diff: 0.11
LEVELS 8000Hz to 9000Hz
dB raw: -53.63 dB dec: -53.68 dB diff: -0.05
LEVELS 9000Hz to 10000Hz
dB raw: -55.14 dB dec: -55.36 dB diff: -0.22
LEVELS 10000Hz to 11000Hz
dB raw: -56.80 dB dec: -57.20 dB diff: -0.4
LEVELS 11000Hz to 12000Hz
dB raw: -58.52 dB dec: -59.09 dB diff: -0.57
LEVELS 12000Hz to 13000Hz
dB raw: -60.24 dB dec: -61.00 dB diff: -0.76
LEVELS 13000Hz to 14000Hz
dB raw: -61.94 dB dec: -62.90 dB diff: -0.96
LEVELS 14000Hz to 15000Hz
dB raw: -63.59 dB dec: -64.79 dB diff: -1.2
LEVELS 15000Hz to 16000Hz
dB raw: -65.22 dB dec: -66.68 dB diff: -1.46
LEVELS 16000Hz to 17000Hz
dB raw: -66.82 dB dec: -68.58 dB diff: -1.76
LEVELS 17000Hz to 18000Hz
dB raw: -68.41 dB dec: -70.50 dB diff: -2.09
LEVELS 18000Hz to 19000Hz
dB raw: -70.01 dB dec: -72.43 dB diff: -2.42

 

[John Dyson]

Oh my....

During further refinement of the release, I found something interesting and IMO, a HUGE improvement.

 

Since there is no spec, I didn't know how many steps the descrambler needed, so all standard freqs were used between nominally 3kHz to 30kHz.   That seemed okay, and seemed to sound okay, but my hearing was really failing.

 

It appears, to get rid of a slightly fuzzy HF (cymbals, etc), both the descrambler 24kHz and 30kHz bands needed to be disabled.   After disabling those two bands, the sound is noticeably more pure.

 

The freq response hasn't been impacted very much (at the +-0.01dB level), but if set reasonably correctly, the descrambler doesn't really affect the average static frequency response.

 

So -- V7.0K will be very noticeably more clean in the HF.   For V7.0J, I didn't even realize the 'fuzz' in the highs.

 

John

 

[John Dyson]

V7.0K is now available, both public snippets and decoder.  Other demos coming in a few hours.

 

https://www.dropbox.com/sh/i6jccfopoi93s05/AAAZYvdR5co3-d1OM7v0BxWja?dl=0

 

1)  better 6kHz descrambler expansion, a little more dynamic sounding.  per user request, and most definitely a 'correction'

2) mistakenly enabled 24k and 30k bands of descrambler, turned off the 24k and 30k bands, producing much less grain.

 

The 'grain' was coming from the 24k and 30k descrambler  bands not having coherent signal to work with.  This was almost like 'hiss' on an FM receiver without a signal coming in.   There is actually an analogy to the FM thing.   Anyway, the 24k and 30k bands were trying to lock onto the signals/distortions probably in the 12kHz range on up.  This really confused the descrambler, causing it to do 'stupid descrambler tricks'.   The grain was actually *modulation* of the lower signals, so couldn't really be filtered out.   By correcting the descrambler, the grain went away.

 

I regret the descrambler grain bug, but I couldn't hear it until today.  It is very easy to fix problems IF I can hear them.  Immediately, when I recognized the grain, I first tried disabling the 30kHz band, but with minor improvement.   Turned down the 24kHz band, still had grain.   Turning off both resulted in REALLY smooth sound.

 

With the two descrambler fixes, I also added about 0.25dB rolloff at 20kHz.   This is a very special kind of rolloff that maintains certain signal characteristics.   The goal is to even better control those little 'distortion things' without substantive impact to the actual signal.   (The filtering is actually more -dB in the inaudble bands, in fact there is NO response on the decoder above 26kHz anyway.)

 

You just might notice an unsettling silky sound, but high detail.  You might be a little worried about less 'sparkle', but that is because the descrambler is no longer sending garbage into the audible bands.   Again, that broken garbage on V7.0J could not be filtered out, unless you roll off at 10kHz or so!!!

 

This should be infinitely better, but definitely a little unsettling.

 

[John Dyson]

Mildly off topic, but also on topic.

If you have an MQA file that is FA encoded (probably most of them, by far), the two bits of noise are hidden by FA decoding.   Think about it...   The lowest bits are essentially noise, and very low levels of noise are almost totally suppressed by decoding. 

If MQA ever does take over, the FA decoding, if the recording hasn't been further processed beyond MQA encoding, will push those extra bits into nothingness.

 

On the other hand, you'll still pay for MQA.

 

[John Dyson]

Been listening to the new decoder version (relatively clear hearing right now), and the results are MUCH more stable sounding.  Very very clean, and more enjoyable.

 

I didn't think that the decoder could get much better than V7.0J, but it really seems that it IS better.

 

Thanks 'private reviewer/user'!!  Correcting the HF expansion spurred on some other descrambler fixes...

 

John

 

[John Dyson]

Just got a bug report about something that is not a bug inside the decoding per-se, but is in the periphery (if it exists.)

 

ADD-ON:   this starts the more disciplined process of actually needing bugs to fix the decoder.

THERE IS NO MORE DEVELOPMENT UNLESS THERE ARE PROVEN BUGS!!!

 

Got a question for those using the decoder:

 

Does adding these two arguments to the command line make the result more clean?

--wia=1.25 --woa=0.80

 

Just add those to whatever command line that you use, best at the end.  (Middle might not work right.)

These were not intended as 'user features', but for debugging/testing purposes.

 

If this is changed, it is NOT to the decoder mechanics, but how the Mid/Side and Left/Right are converted.

 

I'll still be testing tonight, but this is a big let-down if it is true...

If so, I'll fix it asap before the weekend.

 

[John Dyson]

Update, the problem IS a bug, but difficult to hear on most pop.

I only detected a difference on POP, but not easy to claim it was an improvement.

 

The original comment was about classical, so I did some tests with classical music.  WOW, big difference.

Classical is coming through with an even better stereo image when using the now 'V7.0L' decoder, which I just built.

 

Sorry about the update, but I think we want a BETTER decoder, even if it jerks us around a little.

Feedback is still coming, so if the correction is not complete (e.g. it breaks some other decoding attempts), this might require more work.

 

As a proactive measure, I'll be start processing the V7.0L demos, but -- will also be looking for feedback on using the '--wia=1.25 --woa=0.80' on the V7.0K decoder, just to make sure that it doesn't make other recordings worse. 

 

In the very horrible, most worst, most sad case, we might have to deal with two styles of recordings, and I really don't want that.

We have to do what we must.

 

Anyway, without any 'vetos', V7.0L should be available before the weekend, probably +12Hrs if we are all lucky...

 

John

 

[John Dyson]

This is about the command line 'tweak' : --wia=1.25 --woa=0.80

 

The full correction is more than just the changes in the command line.   By changing these conversion factors, some gains inside of the decoder are changed.   This change then requires a change in the internal threshold (the internal version of --coff=), and then required removing two of the much hated 'tweaks'.   So, out of about 5 'tweaks', were are down to about 3!!!

 

The new V7.0L decoder is still being tested/corrected/etc.   So, my guess is that it will take another hour to create the actual, final V7.0L (it is up to V7.0L1 so far.)

After the corrections so far, the sound is pretty darned good.  I am listening to 'Your latest trick' right now, and the cymbals are sounding almost 100% correct.  Still a little disruption, but not bad.

 

There is a lot more verification before the demos can start, but it is very close.

 

John

 

[John Dyson]

One of the kind correspondents  was chatting about finally giving the decoder a try themselves.

Frustratingly to me (and likely to the correspondent), I do think that the decoder is, in the order of importance:

 

*)  too clumsy to use

*)  too clumsy to install

 

Given these two items, as soon as the Usage doc gets updated/completed, I'll start figuring out a GUI.  This old dog is going to have to learn a 'new trick'.

If a non-programmer is enticed enough to try to use the decoder, it really should be 'tolerably' easy to use.   Right now, it isn't easy to use, almost for anyone.

Using SoX with pipes  is probably the easiest way to interface with the decoder,  but then some of the metadata is distorted/destroyed.

That is not a good thing.

 

Simple .wav file I/O keeps the metadata intact, but then the user has to deal with all of the .wav/.flac/.mp3/.opus/.aiff, etc file conversion matters.

That is not a good thing either.

 

It isn't likely that .flac or other format I/O will be added because the decoder I/O code is such an ugly piece of work.   It seems like a GUI might be good enough to improve ease-of-use.

The installation for a non-deep technically inclined computer person is too primitive and inflexible.   Installation needs to be improved, but will probably be a lower priority for now.

 

It is only the last few days that I have been able to think about these kinds of things.  Also have a lot to learn about the user interface matters.

 

John

 

[John Dyson]

The new/prospective version is still being tested.   I have made myself a commitment to make it ready by an hour or so from now, but it doesn't seem ready to upload.   V7.0L has to be 'ready' and the decoding results satisfactory at least an hour or two before I announce it.

 

Right now, I am listening to V7.0L4, and not 100% satisfied.  The much more stable stereo image is GREAT, but when changing the stereo image some other factors got screwed up.   These changes weren't because of a source modification, but a natural result of the Mid and Side signal levels changing, therefore effectively changing the threshold levels.   As a good side effect, also got rid of the LF bump at 200Hz, so we have an appropriately smooth curve for the bass region now.   On V7.0L4, the highs are just a little 'metallic' as if the HF response might be too flat above 6kHz.  It must rolloff to approx -2.5dB at 20kHz, and needs to start at about 7-8kHz, but it seems like the response is a little too extended.  (The rolloff is NOT an actual rolloff, but an artifact of the processing in the decoder.)

 

With the change in threshold levels, the response balance changed.   (several ugly little side effects.)   Anyway,  I know EXACTLY how to find the target response balance now, because V7.0J and V7.0K, however imperfect, do give good, realtively clean references.  Before now, all of the references weren't a perfect match, vinyl ticks/pops, person doing reference rip using non-flat equipment, etc.   Now, we have near-perfect references.   My hearing is not able to reliably compare so far, but it will very likely be usable a few times  today.  Last night, my comparisons weren't accurate, as often happens late at night.  It sometimes takes a few days of hearing cycles to be sure, but it seems like it is 'close', and very step before every release are kept for reference, so correct results are within a step or two from now.

 

During the day today, there is a very big chance of a good/workable release, perhaps 90% that it will be responsible to upload the new version before 3PM my time (hopefully before 1900 to 2100 European time.)    So frustrating, wish I was 10-15yrs younger, the comparisons would have been so much easier.

 

My promise, if at all possible, no more garbage.

 

John

 

[John Dyson]

Working very carefully on the upcoming version, but there is a warning/suggestion.

Before reading this, my apologies for the need for a slight HF EQ redo, but again, the change in stereo image screwed up the EQ to some extent.

 

There had previously been complaints about the sound of the Trumpet on Dire Straits 'Your Latest Trick'.  Specifically previous versions might have a little too much congestion in the sound, or perhaps too much burn (intense HF.)

 

The upcoming release will try to better match the 'burn' 6kHz highs with the FA version, of course with slightly different dynamics.   It is tricky to make the correct tradeoff, but the upcoming release will also try to give better sound/less obscured on the cymbals.   I believe that the cymbals were not intended to be obscured.   This means that the dynamics processing associated with the descrambler needed to be slightly rebalanced, but without increasing the 'burn' too much.  As always, the rules will be followed, unless a really extreme case.  At most, one of the tweaks previously removed might be added back (out of the 5, 3 are left, but 1 might be restored).   This is out of many 100's of EQ steps, many are in 'macros' so aren't all that messy.

 

V7.0L is not ready yet, but wanted to forewarn about the change, and ask the few reviewers/testers to think about the sound of trumpets.

There was no intention to make any changes, but this darned stereo image fix has been a bit troublesome.

 

John

 

 

 

[John Dyson]

This additional delay is going to be small.

The code is open 'one last time', I hope.  The HF EQ was double checked today .  I took advantage of needing to open it up to EQ optimizins for the correction of the stereo image.  Of course, as mentioned above, the thresholds changed, therefore changed the EQ a little.   However, I spent ALL DAY trying different combinations to get nearly the same frequency response, and after all of the probably 5-6 different EQ sequences, we are back to the original plus a few very minor modifications that address a few minor problems that were mentioned.  There was NOTHING that sounded better than what we have been using (plus a few very minor changes.)

 

This is actually very good news, is because after a LOT of work today, there was nothing better than the same basic EQ that we had been recently usage.   In fact, it seemed that the LF EQ was also good enough.   Over the last week or so, we have tested it very well, with acceptable results.   It is stupid to break the code, therefore changes require a very strong, powerful reason.

 

On this release, the forcing factor for the change is for correcting the stereo image and making it more stable.   A side effect is that the highs should be a little more transparent.

I am a little bit worried about the intensity of the super highs, but always doing very careful A/B comparisons with recent versions and the FA original.

 

The final demos haven't started yet, and probably wont start them for about 30more minutes, perhaps an hour if I get distracted.   The wait will be worth it, but this is a 100% effort.

 

Most of the work today has been due diligence to make sure that we really do have the best that I can figure out right now.

 

John

 

[John Dyson]

Status update on the release...

Had to make a last minute change.   I normally have a policy of adding a subtle rolloff, just very subtle.  It's intent is to blunt off some Gibbs effect and isn't really very audible.

However, the new version is so good that the rolloff is probably not a good idea.   That rolloff was just removed in the source before restarting decoding the demos.   The rolloff can still be invoked on the command line.  It will be approx +3more hours before the upload can start.

 

Frustratingly, I have gotten two comments in the last few hours about the decoder being a pain in the 'b*tt' to use.   It has been very enlightening to get comments like this.  The motivation for a GUI is increasing exponentially, impeded only by the last minute bugfixes, the Usage guide, and my learning curve.

 

I do acknowledge that the GUI is critical for almost anyone to use the decoder.  The issue is being studied and WILL BE ADDRESSED.

 

Thanks to the kind people who are telling me about the awkwardness and other usage problems.

 

It might take a few months patience for the GUI, because it is a bigger project than a normal self-contained GUI program.  However, your patience/review of the decoder output and associated comments will eventually result in a useful decoder.   Whenever you have troubles using the decoder, TELL ME, and I'll help and try to resolve any problems on a 1:1 basis.

 

THANKS TO EVERYONE...

John

 

 

 

[John Dyson]

V7.0L is ready.   Decoder has been available for an hour or so, snippet uploads are complete. Other areas are still in progress as of this posting.

https://www.dropbox.com/sh/i6jccfopoi93s05/AAAZYvdR5co3-d1OM7v0BxWja?dl=0

 

This is NO LONGER A TOY.   It is serious-good now.

 

There will be more info later, but you might be mega surprised about the stereo image.   It can be amazing and/or beautiful.  It no longer sounds at all like the FA source material.  Smooth, clean and beautiful.

 

It is now giving the complete spectrum of sound quality that I had intended 5yrs ago.   Everything might not be perfect, but relative to EVERY PREVIOUS VERSION -- the result is perfect enough.

 

I do regret the horrible installation process and the even more horrible user interface.   That will be addressed, and as soon as I can.  The first step is the forever delayed user guide (we won't do a full manual yet.)   I am really gonna have to spend time on the GUI, and it will be a learning curve.   It has been almost 30yrs since I have done anything with GUI, and that was under Xaw/Tk Xwindows.   A long, long,. long time ago in another galaxy far away.

[John Dyson]

I just got some feedback that there might be alittle bit too much presense.   I assume that to be the 3kHz to 4.5kHz region.   It might have a bit more 3k to 4kHz  than the FA version, and maybe should have less.   The general sound is dead-on though.

 

ADD-ON Followup:   with the feedback that was given, it required another listen.   In the upcoming correction, a lot of the high frequencies have less energy available for the descrambler.   The 'sound' is less 3-6kHz, but the high-highs have a more 'sweet' sound.   The response measures pretty much the same.  (sometimes -0.10dB in the higher freqs.)   This response difference is within the margin of error, but there is a very slightly  more mellow & sweet sound.

 

I'll do more investigating later today.    A V7.0M, with exactly the same sound, but more mellow is being processed immediately, and will check when it is complete.  This might be a matter of voting between V7.0M with less 'energy' (like turning down the power, but not louder) or V7.0L with some intensity.   Like usual, the change isn't a 'tweak', but is comprised of removing energy providing building blocks.   Superficially, they might look like a change in freq response, but that isn't what really happens.

 

Below is a response curve in the region that we are talking about.  There *measures* approx 0.1dB variation, maybe meaninful maybe not.   My guess is that there is something about the energy in the descrambler rather than static frequency response.

 

 

 

 

John

V7.0.L:

 

LEVELS 1900Hz to 2000Hz
dB raw: -49.30 dB dec: -49.04 dB diff: 0.26
LEVELS 2000Hz to 2100Hz
dB raw: -49.62 dB dec: -49.35 dB diff: 0.27
LEVELS 2100Hz to 2200Hz
dB raw: -49.95 dB dec: -49.67 dB diff: 0.28
LEVELS 2200Hz to 2300Hz
dB raw: -50.28 dB dec: -49.98 dB diff: 0.3
LEVELS 2300Hz to 2400Hz
dB raw: -50.60 dB dec: -50.29 dB diff: 0.31
LEVELS 2400Hz to 2500Hz
dB raw: -50.91 dB dec: -50.59 dB diff: 0.32
LEVELS 2500Hz to 2600Hz
dB raw: -51.21 dB dec: -50.87 dB diff: 0.34
LEVELS 2600Hz to 2700Hz
dB raw: -51.50 dB dec: -51.15 dB diff: 0.35
LEVELS 2700Hz to 2800Hz
dB raw: -51.76 dB dec: -51.41 dB diff: 0.35
LEVELS 2800Hz to 2900Hz
dB raw: -52.01 dB dec: -51.65 dB diff: 0.36
LEVELS 2900Hz to 3000Hz
dB raw: -52.25 dB dec: -51.89 dB diff: 0.36

LEVELS 3000Hz to 4000Hz
dB raw: -49.18 dB dec: -48.83 dB diff: 0.35
LEVELS 4000Hz to 5000Hz
dB raw: -50.66 dB dec: -50.33 dB diff: 0.33
LEVELS 5000Hz to 6000Hz
dB raw: -51.13 dB dec: -50.75 dB diff: 0.38
LEVELS 6000Hz to 7000Hz
dB raw: -51.56 dB dec: -51.19 dB diff: 0.37
LEVELS 7000Hz to 8000Hz
dB raw: -52.39 dB dec: -52.12 dB diff: 0.27
LEVELS 8000Hz to 9000Hz
dB raw: -53.63 dB dec: -53.52 dB diff: 0.11
LEVELS 9000Hz to 10000Hz

 

 

 

[John Dyson]

Another spree of being slightly overly optimistic.  The decoder now produces TOTALLY STABLE sound, but the expansion isn't quite there.   There are descrambler capabilitiies that I have purposefully not enabled yet.   Maybe it is time.   This would bias the descrambler to lower frequencies, thereby decreasing a perhaps slightly exaggerated HF >6kHz.

 

Wow -- it is REALLY stable though.   Rock solid stereo image.  It sounds somewhat like the FA version, but details are more clear, less blur.

 

I never really knew what this would sound like, but there are very few more steps.

 

PS:  enabled the 4.5kHz band and the result is a little closer to what I had hoped.   Amazingly, it is almost flat after enabling the 4.5kHz and the minor change in the EQ steps.  It usually takes a lot more work, but the EQ sequence has been essentially 'cannonically correct', rather than forced.   Things are a lot easier now.

 

 

 

[John Dyson]

Got much better dynamics behavior.

Enabled the 3kHz descrambler step.   This is a profound change.

 

For example, the 3kHz band isn't just 3kHz, but it is everything between 3kHz to 20kHz, but not in the 6kHz to 20kHz band.   So, when the 3kHz band expands, it expands everything from 3kHz to 20kHz.   Likewise, the 6kHz band expands everything from 6kHz to 20kHz.

 

Common sense says that the 3kHz band will have more energy and do more expansion than any other band, except maybe the 6kHz might give it a run for it's money.

 

The versions up until the next version didn't have a 3kHz band (or previously I thought 4.5kHz was correct).   Since it didn't have the 3kHz band, then all of the expansion/descrambling associated with the 3kHz->20kHz was missing.   It doesn't seem like, at least over the last few hours, that there is all that much of the descrambling effect, but is more of an expander, no matter,  there is still a lot of testing.   This missing 3kHz band is the reason why the decoding results didn't quite have the 'bite' that it should have.  I was greatly disappointed when waking up this morning, listening with my 'good ears' in the morning, and heard very stable mush.   The stability was and is great -- something never achieved before, the there was a missing aspect of the dynamics.   Yea, and I tout dynamics all of the time, but screwed them up.   I know what the decoder needs to do now, but sometimes dont know the settings and don't know what to expect on the output, but just that it should be more stable, more clean, tighter sounding than the original.

 

There were other side effects to enabling the 3kHz band, including a need to correct the HF EQ again.   Interestingly, enabling the 3kHz band allowed a LOT of EQ to fall out, and now the HF EQ consists of some simple, regular patterns.   The side effect of the MF change needed for the descrambler/3kHz change. included a further simplification of the LF EQ.

 

Apparently, the 3kHz descrambler being enabled has greatly simplified the output EQ to the point of almost being comprehensible.   I should have realized this a long long time ago, but never had a 'descrambler' until a few months ago.   Before the descrambler, there was a lot of already learned facts, of course many became incorrect once the descrambler block was added. I held onto the 'old facts' too long, and didn't adequately break away from the old, bad 'reality' to create a 'new' ' more accurate' reality.

 

Frankly, V7.0L, no matter how good in some ways, was a terrible embarassment.

Since the change in the prospective V7.0M is very serious, it requires a lot more testing.

 

NOW, as of V7.0L, we have a totally stable stereo image, generally clean sound with inadequate dynamics.   The inadequate dynamics had some unpleasant effects including the active dynamics above 6kHz would cause a disconcerting imbalance, causing instantanous frequency response imbalances.   This can be ugly, giving a millisecond length shrillness

 

In V7.0M, the dynamics will be improved, hopefully fixed.   I wonder what kind of new bug will appear, what else did I miss?

Maybe, since V7.0L is so close to correct in several ways, but the V7.0L sound might even sound worse than previous, technically less accurate versions.   So, when I write something like 'close to correct', it really means -- the decoder is doing a lot of its work very correctly.

 

SO much for the blather, believe me -- the embarassment has been motivating to remedy the problem.   Hope that this descrambler issue is the last bug.

 

 

[John Dyson]

Well -- the descrambler has been a b*tch to fix.

Several messages ago, I might have mentioned that it works in some ways like an FM radio receiver, with a discriminator-like design.

I am still verifying the correction because this was a very non-trivial change, and requires double checking, triple and quadruple checking.  Only up to 'double checking' so far :-).  Have at least another day of testing and 'hearing cycles before trying another release.

 

This 'discriminator' design conceptually uses a simultaneous 'reception' of several frequencies.   The main focus of the work in the last day or so is to add-in the 3kHz band with an additional associated frequency correction.   * below is an 'interesting' note about using 'tells' when reverse engineering/designing the DA decoder.

 

The challenge was that, after attempting to enable the 3kHz band, the result was reviewed audbibly, but the sound continued to have restricted dynamics.  Additionally,  'enabling' the 3kHz band 'ate-up' the signals for all of the bands, and the expansion would collapse.   Also, all along, there was 'something missing' in the sound related to the descrambler action.   Good news, the problems were found, hopefully the last ones.

 

General description of descrambler bug:

 

The basic goal was to enable the 3kHz, if exists, to improve the perceived 'expansion.  As mentioned above, the initial focus was the 3kHz and somehow it wouldn't work.   Since the design is something like a radio, a good first method of attack is to 'tune' the device.   Tried the obvious option of 4.5kHz.   The attempt at 4.5k results from the concept to avoid the midrange region.   There is some opportunity that the distortion from the scrambling would be more audible, but that worry was unfounded.   The frequency was found to be in the 3kHz region, but the 'tuning' was 110.75Hz in error from the assumed 2990Hz (13.5 * 221.5Hz)  center freq*.   By choosing the correct center frequency AND the strength of the descrambling, the dynamics 'popped'.   The 3kHz-20kHz band started expanding, and the rest of the overlaid sections seemed to work. (Originally, 2990.25 Hz was chosen to match the 3000Hz nominal frequency.   The idea of a non-integral 'harmonic' was wrong, so the subsequent goal is to find out the correct choice.   The freq 2879.5Hz was found to cause the expansion to 'pop'.)   Double checking the other freqs also found that the nominal 9kHz frequency was the incorrect choice.   Instead of 9081Hz, the correct freq is 8860Hz.*   Along with the new freqs 'fixing' the descrambler, the freqs for the nominal 3kHz and 9kHz were changed throughout the decoder.   Keeping all of the freqs in sync was, of course, trivial because manifest constants re used throughout the decoder.

 

* BTW, originally when designing the DA decoder, I found a similar odd set of freqs.  The 0-80Hz band was actually 74Hz (221.5/3), 2880Hz, and 8800Hz.   I found these freqs by 'listening for tells', not  knowing the 221.5Hz rule at all.   This was approx 5-6yrs ago.   This is definitely evidence (not proof) that the 'listening for tells' technique is very helpful.  Listening for tells is definitely is not 100% reliable, but is very useful when the general frequency region is known, but the precise freq is unknown.   Reverse engineering the schematic was approx as accurate as 'listening for tells'.   The values of just below 80Hz, just below 3kHz and just below 9kHz had to be verified because of non-theoretical/limited gain transistor designs along with 1% component accuracy.   "listening for tells' was just as accurate as reverse engineering the schematic or moreso.  (I used LTspice for estimating the freq, along with a cookbook formulaic calculation.)

 

Since the tuning frequency is obviously more critical than using Q=1.414 and Q=2 in the mix might imply, the other frequencies were also tried.   Fortunately, another error was found, where the previous choice of 9081 Hz was wrong, and 221.5Hz lower was also correct.   The result of this correction is more crisp highs, because the overlaid 9kHz band was missing.

 

For '3kHz', the 'discriminator' needs to be tuned with a 221.5Hz deviation, while the frequencies above 3kHz (6kHz on up) work best with a deviation of 221.5Hz * 2.215.  Of course, when determining these values, a bunch of listen/change/listen/change/listen/chance tests were needed.

 

General theory of descrambler behavior:

 

The discriminators consist of two Q=2 high pass shelf filters, at freq + <deviation freq> and freq - <deviation freq>, plus a central high pass shelf filter at the center freq, with Q=1.414.   The intuitive effect of the descrininator is to replace the Q=1.414 filter with Q=2 as the frequencies deviate from the center, thereby 'amplifying' the signal based at the center freq as it deviates from the center.   The effect is to 'spread' the signal  energy in the frequency domain, which 'sharpens' the edges in the time domain.   When decoding, the design would be inverse, thereby making the energy in the frequency spectrum peakier, which spreads the signal in the time domain.   Basically, the 'encoding' side acts something like a super-duper fast peak limiter (or more intuitively accurate) peak 'rounder-offer'.   On the other hand, to decode, the 'rounded-off peaks' are restored to their nice sharp characteristics.

 

Since human hearing generally cannot distinguish between the wider energy signal vs. narrow version except for the sense of peak limiting, the goal of 'smoothing' the signal during FA *encoding* is met, along with the result not being perceived as distortion.   Also, the choice of a narrower 'deviation' at 3kHz is obviously related to the more acute sensitivity to spectral energy at the lower frequency.

 

 

 

 

 

[John Dyson]

Having a MEGA bad trip trying to 'flatten' the response.   Getting a 'measured' flat response with the descrambler enabled is fairly easy, but the problem is in choosing which of the EQ schemes is canonically correct.  Also, the 'measured' response is dependent on program material used to drive it. This kind of necessary decision is tedious to resolve, requires patience and is incredibly error prone.

 

Since the descrambler has a variable frequency response,  I decided to disable the descrambling entirely, simply putting it into null mode.

 

When disabling the descrambler's  'mojo', that is, disabling the discriminators, yet keeping them 'in circuit', the descrambler seems to be providing some functionality that I didn't predict.  However, when thinking about it again, it is easy to understand.

 

It 'seems', after measuring the beahvior with disabled 'mojo', that the descrambler is providing very significant noise reduction without having an attack/release time.  This is still emotionally difficult to accept, even after thinking about it for a while -- heres the results:

 

1.5dB unweighted NR at 6kHz

5dB unweighted NR at 9kHz

9dB unweighted NR at 12kHz

14dB unweighted NR at 20kHz.

 

Obviously, these are assuming that the NR is flat and fully realized.  My guess is that the amount of actual NR might be about 1/2 those dB values.   Bottom line, my original assertions that the descrambler provides peak restoration was uninformed and incorrect.  Instead, it appears to provide the entire spectrum of expansion in the HF region.

 

I am doing further investigation, but when measuring a frequency response of -14dB at 20kHz and -5dB at 9kHz when the result is audibly flat tells me that the measurement isn't applicable for my purpose.

When re-enabling the discriminators, then the frequency response pops back up.

 

Wierd stuff.   How do those guys/gals (the original designers) think of this stuff?

 

[John Dyson]

About the descrambler:

 

I must have been trying too hard.

The HF EQ:   simplified it to absolutely as simple as possible, inserted the descrambler running the full -20dB to +3dB gain variation, set the deviations on each discriminators correctly...

Eureka!!!

 

I kept trying to 'limit' the gain changes, 'flatten' the response.   There is absolutely NO flat response that I can measure, and didn't even try to match the LF/HF gain,   The sound just came naturally.

 

Pretty impressive sound -- REALLY.   Alive.

 

I sure hope this is the answer.   The sound hasn't been very satisfactory until now.   It sounds *good*, just gotta make sure that it sounds *right.*

 

This problem all along -- once the correct design was derived, I never trusted the results.   I have lost a lot of confidence, but if I had kept some amount of trust in myself, this result (or minor variation) might have appeared a long time ago.

 

John

 

[John Dyson]

Just told someone else privately, but thought that this announcement is worthwhile:

 

ALMOST ANYONE, and I mean ALMOST ANYONE will be able to distinguish FA recordings from non-FA recordings in V7.0Q (yep, major changes.)  The difference when using the new version of the decoder is so profound that non-FA sounds unlistenable when trying to decode.

 

The processing is aggressive (in the American sense), and will NOT work with non FA.   The attack/release time for the 'pseduo-expansion' is infinitely fast, and the gain change appears to be -14dB to +3dB.   The noise reduction is more complete than before.  The original 'dynamics decoder' layer only  works at fairly low levels below -30dB.   This is why the 'dynamics decoder' only made subtle improvement, however valuable that that improvement might be.   However, the 'descrambler layer' always works and is not subtle.

 

I have some old recordings that the old decoder versions would still let hiss leak through.   The new version with the descrambler has such fast attack/release that the hiss simply goes away.   After listening to the new decoder, I suspect that many people will hear extreme deficiency in the original FA version...  I am hoping to make the demos available ASAP, but I don't wan't to make a mistake on THIS release.   There might still be minor nits, and we need people with GOOD HEARING to review it, but ANYONE will be able to hear the improvement and will show most listeners that the decoder concept is unquivocably worthwhile.

 

I am wondering of a 'poor persons' verison of the decoder might be created, where it might provide the work of the descrambler only, and be practical for a DAW plugin.?  There is some testing to verify if/if-not it might work.  The descrambler might actually require the output of the dynamics decoder to function properly.

 

John

 

[John Dyson]

Learning a lot more about the descrambler.   Very interesting processing 'engine'.   The constituent elements are very simple to understand, but the combination can do some pretty amazing things.   Been having troubles getting control of it, because it can do very aggressive processing (I really mean 'aggressive' this time) without extreme audible side-effects.   Amazing, really.

 

It has taken a few hours to find the exactly correct and modest settings in the most impactful frequency regions, but the result is still profound in two ways.  With the correct settings, the upward expansion is small, but important.   The downward expansion is the amazing part.  It really can give more than -10dB downward expansion.   The noise reduction (hiss reduction) is a very helpful addition to the hiss reduction in the original decoder components.   The better hand-in-glove inverse of the original compression helps the sound a lot.

 

The result is a more '3d' sound, perhaps more than the recent stereo image improvement.   I believe that without the stereo image corrections done a few days ago, refining the descrambler settings would have had less positive impact.

 

 I am still trying to find the correct settings for the 24k and 30k bands again, finding that they might be helpful for the highest freq transients.  (30kHz band still has impact as low as 12kHz.)   The most critical, of course, are the 3kHz, 6kHz, 9kHz and 12kHz bands.   18kHz has more importance than 24kHz for two reasons -- 18kHz has significant effect down below 12kHz, and there is energy being absorbed from the lower bands, thereby better matching the encoding process.   Even at 18kHz, without absorbing it's correct energy in the 6kHz to 9kHz bands, the result can be some 'surging'.

 

Not fully understanding the descrambler, my guess is that some aspects might be helpful for my brute-force anti-fog mechanism.   The anti-fog works really, really well, but takes so much CPU that intuition tells me that there might be a more efficient way of doing the same function.

 

The current project focus is ONLY on finishing the descrambler though.  I thought it was finished a week or so ago, but when I kept getting reports about some flaws, the problems seemed to be pointing to the relatively 'new' descrambler.   Other improvements are coming later, the usage doc and the GUI are more important than almost anything else.

 

 

 

 

[John Dyson]

Release is getting closer.   Had another delay, another game being played with the audio.

However, after a lot of experimentation about 'which choices are correct', I think that the descrambler is much more of a genius invention than I had originally even thought.

 

Note the discussion about descrambler expansion earlier on.   It does expand upwards/downwards, in both the upper midrange and the highs, but in an incredibly odd way.

Think of a 'see-saw'.   The upper midrange is boosted when the HF is decreased and vice versa.

 

The descrambler expands the upper MF and HF, but in a way that the dynamics of both are improved, and they are both 'squished' together rather than being 'squished' in one direction.  Postive genius.   Unbelievably tricky concept.  First technical design that i'd call a 'hat trick'.   The pivot for the 'see-saw' appears to be around 3kHz as expected.   The see-saw changes the 1kHz to 2.5kHz range by +0dB to almost 3dB.    The 4.5-20kHz region changes by 0dB at 3kHz to 0->-1.5dB at 4.5kHz, 0->-3dB at 6kHz, 0->-6dB at 9kHz up to about 0->-10dB at 20kHz.   The see-saw thing is intriguing.

 

The descrambler expansion is impactful over a wider range than I had originally surmised, and is very good at bringing vocals that had been 'buried' along with accomopaniment up by a dB or so to be more forward, more 3d.

 

So, we have the descrambler down-pat, and also of the various possible configuration choices, I have chosen the most simple, most straightforward with the minimal 'adjustment' from the simple concept.   Totally as-is sounds pretty good, no complaints by me (with generally poor hearing.)

 

Found another nit -- another game being played with the audio.   As you probably can guess, the FA compression can make sibilance disappear because it is so fast and is very impactful in the 3k to 20kHz region.   There are some magic frequencies for sibilance, perhaps the most important frequency is in the 4.5kHz to 7.5kHz range, with 6kHz being the center.   There was an interesting pattern where after simple decoding/descrambling, if the 'intense' sibilance is passed through an equalizer with a specific EQ pattern, the sibilance starts sounding 'normal'.   For example, perhaps the absolute worst case has been 'Carpenters' in general, and 'Top of the World' being egregious with so much sibilance that it is unlistenable.   After passing through the new EQ, the sibilance normalizes.   The correction isn't just for sibilance, but a lot of other kinds of 'details', where excesses are suppressed, and missing 'tinkles' are restored.   Sometimes, sibilance was suppressed before the 'sibilance restorer', however sounds 'normal' after it was added.   Likewise, ear-burning sibilance is corrected back to normal.  Yet another 'encryption' against the audio.

 

The release now has the 'see-saw' configured descrambler and the new  'sibilance restoration' block installed.

Most challenging for me:  Is the 9kHz rolloff needed?   What kind of alternative to the straight single shelf?   This decision is not made yet.

 

There are several possible 9kHz EQ configs, but with the 'see-saw' descrambler, it doesn't *seem* like an explicit 9kHz rolloff is needed.   This determination will take a few 'hearing cycles' to choose whether or not to add the 9kHz EQ back in, or even if added back in, what kind of config?

 

It is actually seeming that the 9kHz EQ will no longer be needed, which in an intuitive sense, a good thing.