'FeralA' decoder -- free-to-use

[John Dyson]

1 hour ago, fas42 said:

Just to give some immediate feedback, John ... again, the Super Trouper snippet. The voices intro lacks sparkle and air; and the following instrumental run sounds, 'artificial' - the following seems OK ... this was comparing it to the RAW sample, which I downloaded some time ago.

 

If you want me to elaborate more, let me know ...

Thanks,

I understand there is room for criticism on these test/demo examples    I was trying to show the detail that digs deep, but doesn't sound gritty.  Also, you have verified a concern about excessive 'airiness' that I have been perceiving also.  I didn't know if the problem was real, and you have told me that there is still a problem.  (The problem is related to descrambling in the 3kHz region.)

 

ADD-ON:  per the comment about using 2.75kHz instead of 3kHz for the base frequency -- I gave it a quick try (yes, I can check these things in seconds), and 2.75kHz seems to 'pop'.  I am not fully sure yet, but responding to you about the 'realness' issue had got me thinking about possible solutions.   THANKS again...  Just might have found an improvement, if not the solution....

 

ADD-ON:  Also, noticed that thet HF > 3kHz needed more dBs.   Frustratingly, the descrambler totally screws-up the idea of 'flatness'.   Will explain in the future, but suffice to say that the descrambler is a whole bunch of trouble.

 

For something that I otherwise FINALLY fully understand (the architecture of the FA encoding/decoding process), the descrambler has been a final 'monkey wrench' that was unknown to me until about 1month ago.  THIS IS NOT AN EXCUSE, but I am relieved that you perceive a quality problem that has been bothering me a lot also.  I haven't been sure that the problem really exists!  YOU have verified that the problem needs addressing.   I have REALLY needed & appreciated  feedback on this matter, and I felt that it was best to avoid giving 'hints' about potential flaws.   Normally, I try to ask for help in private communications, but wanted to, in public, show some actual activity also (not just me blathering about stuff being done.)

 

  I had originally thought that the descrambler didn't need additional EQ, and it actually doesn't measure like it does.  Just wondering that I missed the need for EQ, or that the 3kHz region is messed up.   There is definitely something weird going on in the 3kHz region -- I know it seems like I am rambling, but am a little confounded/confused about what is going on.   3kHz is 2/3 of the reason for the delay right now.

 

The descrambler 'thing' does very significantly improve the detail by straightening up the signal, but there are side effects that I still don't fully understand and haven't determined the correct settings.

 

I am allowing myself only one more day to figure out what to do about 3kHz and find the canonically correct settings.  If the descrambler continues to be vexing, then either it will be made optional, or will be running in a safer, without 3kHz mode.  Disposing of the descrambler issue for now will allow doing a release replacing Friday's mistake in a day or so.

 

3kHz is so frustrating that tomorrow, I might also be taking 2.75kHz and 4.5kHz more seriously to consider as the base frequencies.  More frequencies mean a lot more variables, and the 4.5kHz frequency actually violates the 'rules', while 2.75kHz is within the set of rules, but is a bit of a problem for other reasons.

 

Again, thanks for recognizing the same problem that has worried me, but haven't been sure if my tone-deaf perception was fooling me.

 

 

 

 

[John Dyson]

Good news:

Summary is that I plan a release tomorrow afternoon/evening, delayed only because of logistics, otherwise might have been tonight instead.  Som additional testing/verification will be done during the 1day delay.

 

The 'liveness' type problem is fixed (I think, if I had understood it).   The 'liveness' issue was caused by a mistaken join between MF and HF, where the signal was flat instead of a rising response needed to match expected *measured* sound and to be 'flat' without the descrambler enabled.  When the descrambler is enabled, measuring the spectral average signal match between FA and decoded becomes nearly impossible.   An estimate can be made because of the small approx 1.5dB increase in peaks in the HF region.   Because the descrambler is well behaved, and the sound is as expected, then it appears that the descrambler will be enabled by default.   The descrambler can be disabled, and if there are complaints, we can test by disabling it.  I'll probably document the 'disable' when the usage doc is updated soon, but shouldn't be needed.

 

The bad news...   Last night, ran the demo decodes, but mistakenly left the pre/de-emphasis enabled that compensates for the missing descrambler.  However, the descrambler was enabled, so certain crescendo type sounds became overemphasized.   That problem is fixed, and right now I am listening to one of the examples where the 'excessive expansion' type sound was previously manifested.

 

The descrambler is set at the most trivial setting, where it is as if the original FA designers set it to a maximum type setting with no emphasis for any one audio band.   Basically, a 'vanilla' setting.   Also, the 'test run' last night used the 2750Hz setting for the first band instead of the previous 3000Hz setting, with improved results.   This was mentioned in a previous post.   Before the final run, I'll do one more double-check on this matter.

 

The sound of the descrambler is pretty much neutral, and without A/B immediate comparisons is often difficult to detect.   The difference is more in 'feel' when listening casually.   There are qualitative differences where some recordings end up being more smooth, others have more clean dynamics.   There are perhaps a few regressions, but it seems like it is more of an 'alteration' from expected sound instead of an actual bug.

 

Like usual, sometimes expected times can be delayed, especially if during checking/verification if a bug is found.  A full 'listen' with correct settings hasn't been done yet, but even with the mild defect last night, the results were pretty good.

 

 

[Confused]

11 hours ago, fas42 said:

Just to give some immediate feedback, John ... again, the Super Trouper snippet. The voices intro lacks sparkle and air; and the following instrumental run sounds, 'artificial' - the following seems OK ... this was comparing it to the RAW sample, which I downloaded some time ago.

 

If you want me to elaborate more, let me know ...

 

I presume you are still checking the snippets via your laptop speakers? Or have you found a way now to feed your PC sound to the Edifiers?

[John Dyson]

32 minutes ago, Confused said:

 

I presume you are still checking the snippets via your laptop speakers? Or have you found a way now to feed your PC sound to the Edifiers?

My problem is that my hearing changes without perceiving a difference.   The decoderis capable of being beautiful, but I am working here alone with incredibly unreliable hearing.   Normally, I use very wideband headphones that go far above normal hearing and reach almost to DC (really -- very low freq.)   They are also relatively flat, but my hearing is almost without any rhyme or reason.  

There is finally a method (non sine based) that the decoder can be proven flat within a few tenths of dB, but the descrambler does weird things, including changing the peak to average ratio.   These changes render normal measurement methods almost useless.

 

As an engineer (much more qualified than most in DSP, CS and EE), I am used to working very hard and intensively, but my hearing diminishes within about 5minutes of getting up from a laying-down position.   Since I really do not like working when laying down, there is usually only about 5minutes of reliable hearing every 4-6Hrs.   This is frustrating, and been a source of embarassment.   If only there was a well meaning person with whom I can communicate in pure technical terms, and also having good hearing -- the project wouldn't have been delayed and screwed up so often.   Sadly, not everyone has been well intentioned, and been too skeptical of the effort.   HAPPILY, there have been a few kind souls, one who has seen what I have done in the past, who still helps the project.   Those who are too skeptical don't know what they are missing, but there is no way to convince those who 'already know everything'.

 

So far, the decoder without HF phase descrambler is very much proven, but the phase descrambler has been problematic.

Maybe I should have terminated this phase (REL5) when I had intended, but when I detected the scrambling in the consumer recordings, it seemed to be best to resolve the scrambling issue.

 

Truly, the quality of consumer recordings is JUNK compared with an original mix, and the decoder project is attempting (mostly successfully) trying to solve it.   Alas, at Bell Labs, I always had at least one co-worker working intimately with me on the project...   Myself being the wild, slightly off-center creative person, and the other also being very creative, but with both feet on the ground.   Usually, this mix has produced amazing results, but I am without the 'solid footing' of an equally qualified engineering person...

 

Frustrating as h*ll.

 

[John Dyson]

Because the descrambler is so difficult to prove audibly flat, I'll be releasing the decoder without descrambler.

The decoder still improves the sound, but there are obvious, lingering problems from the FA version of the recording.

 

The decoder itself has been very accurate in the lab for the several weeks or so, but been mired in optimizations and finding the sound quality problem with the 'smear' in the sound.

 

I think that staying in research mode is not helpful at this point.  It will take a day or so to shift gears, copy code from earlier verisons, and retest/verify the response.

 

John

 

 

[John Dyson]

Found a version that is flat in the highs and lows (well, as flat as the measurement technique can measure), and has an early descrambler in the HF EQ code.   If the degenerate descrambler ends up being a problem, then it will be replaced by the equivalent simple EQ.   The code has been well proven and sounds good, plausibly undoes the FA encoding.   Some tests show that it ends up being very close to correct, but perfection is only an emotional release and not an objective statement.   So, I won't call it perfect, but appears to be fairly accurate.

 

The attack/release, etc has been carefully refined in recent weeks, and the expander effects are *gone*.  Most of the infrastructure bugs have been reduced to almost nothing. 

 

The descrambler is very easily removed, and right now I am biased towards removing any vestige of a descrambler.    The descrambler project needs more research, and it has more tentacles than I had orginally thought.  The major difference between the V6.0.4X base EQ code and the eventual V6.0.6XE/later versions will be removing the 'descrambler and using the 221.5 Hz basis for EQ instead of 250Hz.   This would be a micro level improvement over and above the earlier EQ, even if some  'new stuff' isn't going to be utilized.

 

Adapting the old code into an updated infrastructure  is almost a total EQ redo from where the project was sitting.   Even if the behavior seems to be the same, it will be very necessary to cycle through several 'hearing cycles' before being confident of the results.   It would be 2days at the earliest for a release, but might show/upload  some 'status' demos for gaining helpful feedback.   Gotta double check my bad judgement about hearing things!!!   A few very helpful people have given feedback on status uploads, publically and privately, and it is very much appreciated (e.g. @fas42)   Others, who will remain private unless explicit permission is given, have also helped and encouraged.   Just a real thanks -- I need encouragement.

Thanks again, and really trying hard.  I must be more disciplined about the reliability of my hearing, but it is so easy to be reckless while ignoring my hearing limitations.

 

 

[fas42]

6 hours ago, Confused said:

 

I presume you are still checking the snippets via your laptop speakers? Or have you found a way now to feed your PC sound to the Edifiers?

 

Yes. Those speakers are essentially ordinary quality headphone drivers, which just happen to be sitting in a laptop casing. Which means they have no bass below about 200Hz - something which one has to take into account. But certainly good enough to highlight differences, and which make flaws of audio systems in YouTube captures easy to pick .

 

Perversely, an ordinary quality playback of a track makes it easier to pick possible issues ... why? Because, a much higher standard of SQ allows the brain to bypass technical problems, and relate directly to the music making. Which is not what you want, in this situation ...

 

A similar thing to how in recording studios they often have a pair of very ordinary speakers, to make sure a mix is going to sound balanced when heard over everyday sound replay devices.

[John Dyson]

t

13 hours ago, fas42 said:

 

A similar thing to how in recording studios they often have a pair of very ordinary speakers, to make sure a mix is going to sound balanced when heard over everyday sound replay devices.

 

I agree.   Even with the best, most 'flattest', least distortion device for signal verification, can encourage biased result when playing through less perfect devices.  It is important to avoid extreme signals, even if correct.   A true bass signal might overwhelm normal speakers, or true dynamics might force some playback situations to either clip or portions of the signal might have too low levels.

 

Certain kinds of processing can be used to modify the extreme signals into something that can be played on all kinds of equipment without problems.   Testing on different speakers and different headphones (both) can avoid many kinds of mistakes.  By learning the hard way, headphones can be profoundly different as much as speakers being prfoundly different.

 

'Mixing' seems like it can need real expertise.   Starting from 'scratch' with a raw signal not based on actual instruments seems daunting.  I have done natural, stereo recordings of actual orchestras, but that is easier because it is directly based on actual sound.   Electronically created sound is basically starting from scratch, and 'reality' needs to be artificially created.   Perhaps some people have a natural skill to be able to create a plausible reality, similar to an artist creating a painting from memory.

 

 

 

 

 

[John Dyson]

Oh my....

I almost, within about 1/2 hour, of asking:  Is this it?  Is this they way that it should sound?  Is this right?

 

The problem is that it ended up with a small 'honk', and the 'honk' happened after the 2nd to the last change, basically lining up the 2.75kHz with 3kHz descramblers. (Yes, found out that two are needed in the 3kHz range.)   The problem isn't with the descrambler, but instead the EQ before the descrambler (mostly with 2.75kHz EQ.)

 

The first hint was an irregular sound/balance that I could only detect when my HF hearing had diminished.   Next, ran a spectral response test, and found that the 1kHz-1.5kHz and 1.5kHz-2kHz regions aren't balanced.  It seems like these two regions should be balanced within about 0.1dB, but they were 1dB different.

 

So, the general sound is so good that the 'Is it it?' question is coming soon, probably within a day or so -- REALLY.  I was hoping for immediately after this decoding session would complete.  One part of this frustrating news tidbit is that a small change to make the code more symmetical borked it.   I should have double-checked that balance in the 1k to 3kHz region, very very important.   Excess differences in that region seem to potentially cause an undesired emphasis in the 500Hz to 1kHz region --HONK.

 

It is REALLY close to being ready.

An additional 'good thing' is that the higher quality modes have been corrected.   Since we found that the anti-distortion cannot be allowed to funciton with sideband cancellation below about 1kHz, then the longer Hilbert transforms are unusable without some changes.   Instead of depending on lower frequency cancellation to improve quality, the window functions become more advanced as the transforms get longer.   This more effective window function decrease the LF response in the Hilbert transform, but deepens the window, also making it easier to 'hit', so that the effectiveness of the anti-distortion is increased.   The result is that --xpppp really does work better than --xpp.   However, very importantly the decoder is so very good now that using '=max' quality mode modifier causes more perturbations in the signal than what normally exists now.   Mathematically, the decoder is now much better, but still might have some nits here or there.

 

Hopefully, IS THIS IT? will be asked in the next day or so.   Note that IS THIS IT? and a positive response means that the usability matters become highest priority.  (Including at least tolerably good docs.)

 

 

 

[John Dyson]

The next release will be an experiment, hopefully the ultimate release.  It will be coming in a few hours.   The demos are complete, and doing the reviews right now.  Unless I can determine that the 9kHz thing below is wrong, the release will be coming in an hour or two.  If the 9kHz choice can be determined wrong, I'll do a quick redo, and will take 3 more hours.  (Not decoding at full quality, goes much faster.)

 

ADD-ON:   after noticing my hearing is cleared-up right now (works really well sometimes), I am pretty sure that the 9kHz +3dB mentioned below should be removed.   Therefore, because of being too anxious to send this out, I jumped the gun.  The corrected releasae  will be delayed another 3 Hrs.   The private reviewers area has the snippets right now for V6.0.7A, and if you think that it sounds good with the 9kHz +3dB, tell me immediately and I'll finish distribution.   The public demos will also have the snippets available in a few minutes.   Feedback about this will be very helpful, no matter the source!!! (The sound of too much 9kHz is a sense of 'grain'.)

 

Here is the experiment:

Normally for EQ like this, 2nd order EQ would be intuitively used in this situation.  As a testing mistake (truly), a special form of 1st order EQ was used, and a 'keyhole' effect was noticed.   This is REALLY strange, but 1st order EQ used in the EQ situation being discussed would normally produce a terribly dead sound.   However, if the EQ is 'just right', the 1st order EQ being used has a window where the EQ is really good.   It isn't generally* too bright, not too dead.   Now, it is for the ultimate testers to make this decision, including especially the actual testers/helpers/reviewers.  *9kHz EQ might be erroneous as described below.  IMMEDIATE fix avaiable, just that I cannot reliably choose.   *EVERYTHING* else is cleanly reverse engineered or a required lock-in.

 

I do NOT know what is going on, and will require some math (or Spice emulation) to understand what is happening in this EQ, but it SEEMS to be really good.   Not having to 'fight' the EQ, but instead use the very special EQ config  (it is a 'delta freq' scheme, will explain if really interested.)

 

This upcoming version has *full* descrambler per what I can suspect the descrambler really ended up being.

 

The major caveat is that there could be an error in a middle EQ step, but that error might come directly from my hearing problems.  (It is whether or not to use a +3dB Q=0.8409 EQ at 9kHz).   Everything else is a perfect lock-in, not dependent AT ALL on the precision of my hearing.   The 9kHz +3dB can be removed INSTANTLY, and is one of those few things that can encourage me to make the wrong choice.   There is a design characteristic that makes the +3dB 9kHz EQ very likely correct.

 

(Demos just finished converting to .flac from full FP quality .wav...   Will start doing the transfers and build Windows version in abt 1/2 Hour to 1Hr.

 

[John Dyson]

The upcoming version later tonight  will make the last seem very embarassing.

Some potential tells went away, therefore really were tells.   This is a good thing.  (A lisp on one of Linda Ronstadt's recordings.)

 

The major caveat is still the super HF frequencies which manifest as perhaps +2dB at 15-18kHz.   Whether or not these are erroneous or not, that is for others to tell me.

The descrambler is working in full force, and material that had muffled cymbals (e.g. The Carpenters), now have normal sounding cymbals.

Other material which doesn't have muffled cymbals for one reason or another are not overenhanced.  There is no overenhancement other than potential HF emphasis (trivial to fix.)

 

Again, the caveat might be a 2nd order type excess of perhaps +2dB at 15-18kHz (OR NOT.)   I promise that the EQ can quickly be changed.  Perhaps I should have created a mode switch, but since the demos don't have a 'mode switch', the 'switch' in the decoder doesn't help the issue for demos.

 

The previoius problem in the earlier posting that I had assigned as being about 9kHz wasn't fully about only 9kHz  The 'keyhole' was missed a little bit at the 3kHz range, and made the grain much worse than it should have been.

 

The decodes are only about 20% complete because the correction was bigger than I had estimated, and tried to be careful when testing (actually laying down so that my hearing is a little better.)   Even then, it is very possible that the HF would be +1dB at 10-12kHz, +2dB at 15-18kHz which might be in the output, or might not.   (there are EQs at 24kHz and 30kHz, which might need to be -3dB each with Q=0.8409.)  I don't know if they are supposed to be in the EQ chain, but are architectural consistent.  Using the EQ seems to deaden the sound too much, and there is no *requirement* that the EQs be in the chain, just that they MIGHT be needed.

 

After this, I'll be listening to the demos as they come from the decoder until the GOOD demos are ready to process and upload.

 

 

 

[John Dyson]

V6.0.7B is ready, and almost all choices for clamping down on the HF EQ, based purely on a design pattern,  were made.  

 

https://www.dropbox.com/sh/i6jccfopoi93s05/AAAZYvdR5co3-d1OM7v0BxWja?dl=0

 

This is the first release where I have 100% assumed that my hearing is not to be trusted, and follow the design pattern except for two very, very limited cases.   I doubt that the decisions the two other EQ will make much of a difference except for <20yr olds and dogs :-).   It is possible to trim a very slight additional HF, but it seemed like a bad idea to trim any more HF, even ignoring my really bad hearing.   My hearing was NOT used almost at all to judge the response balance.   This version is based purely on a design patter, not on any judgement (except for listening for the keyhole effect, which is a very profound tell.)

 

 

The full, absolutely complete descrambler is enabled,  the descrambler is totally filled out.   The *really interesting* EQ sequence using the special 1st order pattern really did create a keyhole effect.  Really strange, but any deviation that might even assume an increase in HF actually decreased the HF.   It is very good that I did a lot of research on esoteric EQ methods, because the techniques allowed producing this release.   The descrambler is 100% integrated with the HF EQ, and impossible to separate without making the EQ totally wrong.

 

You *will* notice a very difference sound with absolutely NONE of the sparkle of FA recordings.   If you take a mic, recording something and play it back, there is NO sparkle like FA has.   This same lack of sparkle exists on the decoded material, yet should have all of the highs on the original mix.  (perhaps might have too much HF.)

 

Have fun.

[fas42]

Quick response, John ... Super Trouper now is fine, I don't have anything to complain about. Then after another, also tried one I hadn't looked at before, Warwick, I'll Never Fall in Love Again - again, no issues.

 

The middle one was Dreamer - which sounds so different from the RAW I have; the DEC has a lot more impact, in a positive sense. The one area I would mention here is in the vocal - when he goes low, there's a suspicion that a bit more spittle is involved than one would want - may not bother some, but something to perhaps consider ...

 

Hope this helps ...

 

 

 

 

[John Dyson]

55 minutes ago, fas42 said:

Quick response, John ... Super Trouper now is fine, I don't have anything to complain about. Then after another, also tried one I hadn't looked at before, Warwick, I'll Never Fall in Love Again - again, no issues.

 

The middle one was Dreamer - which sounds so different from the RAW I have; the DEC has a lot more impact, in a positive sense. The one area I would mention here is in the vocal - when he goes low, there's a suspicion that a bit more spittle is involved than one would want - may not bother some, but something to perhaps consider ...

 

Hope this helps ...

 

 

 

 

Thanks,

I'll give Dreamer a look-see and figure out what is going on.   There could still be some issues (bugs) , esp given the descrambler is very new.  The descrambler is loosely based on an earlier version, but is filled out with more 'frequency balance' methods.

 

Initially, when thinking about 'Dreamer' spittle, too much descrambling ends up being expansion, and it could cause some negative effects.   Also, there is still a potential EQ problem between 3kHz and 6kHz.  I was just thinking about it after the release (uploads/etc.)   The need comes from a complex thinking process, but is hinted at by the frequency response caused by the descrambler.   I *will* look into the 'spittle' matter, most definitely.   Tonight is a night of rest, but am very, very very very motivated to look into the 'spittle' problem that you noticed.

 

(If the rest of this note seems like a 'victory lap', it truly is.   Perhaps, instead of a 'victory lap', it is the 'last lap' before the finish line.)

 

Even though many previous comments are effectively blaming all previous versions on my hearing, blaming solely on bad hearing wouldn't be a totally honest statement.   A whole chunk of technology has previously  been missing from the decoder. The likely correct or almost correct EQ is impossible without the descrambler, so ANY previous version couldn't have been quite as good as this one.   A lot of previous HF problems came from vain attempts to use simple EQ techniques to fix the character of the sound.   Of course, the descrambler is a kind of EQ, but a very subtle and non-traditional kind of EQ.   Some of the earlier frustrated eccentric EQ methods ended up being vaguely reminiscent of the more complete descrambler design,  but direct intuition about raw EQ wouldn't likely have resulted in the same thing as the descrambler.

 

If any perceptual 'tell' that saved project, and kept me (us) from additional years of frustration, was finally being able to hear the need for the descrambler.   Previous attempts to recover the HF dynamics ended up in frustration and general loss of confidence in the project.   I still do not know how the idea of a descrambler appeared in my mind, because it is totally different from any kind of EQ that I had previously thought about WRT audio.   There are similar video techniques as the descrambler, but the method used in the decoder is strongly different from the video approach.  Maybe my mind gained access to the cosmic consciousness?  (Humor intended.)   Truly, I haven't done any research that might have reminded of the scrambling technique, it just appeared to me -- oh, the wonders of the human mind.)

 

Looking for more feedback, and we are so close to perfect that clean, accurate criticism like from @fas42 (and other private AS reviewers)can much more easily be assimilated to further improve the project.   Pure technical comments aren't needed, or sometimes even helpful.   Those reading previous discussions a year or so ago, too much technical description can be very misleading and actually increase frustrations!!!!   Pure technical descriptions really need to be spot-on, because they can convince too quickly.   I really like the hybrid of technically informed intuitive comments.  Stuff like "sounds like too much 6kHz" is very helpful and just technical enough.  My intuition right now is that if the 'spittle' is an eq problem, it is something related to the 3k to 6kHz range, but is partially informed by what I know is going on in the program itself.

 

Admittedly, the decoder previously had serious problems that would have required detailed and very importantly VERY accurate criticism that would be difficult to actually communicate remotely.  This design effort, especially earlier on, would have been MUCH better if two people with different skills were working together in person.  That said, without kind encouragement and helpful comments, very often on AS, over the years, the decoder would have been a continued frustrated effort.   I have to thank our missing friend Alex again for tolerating a lot of frustrated discussion, and sadly he didn't get a chance to see the recent progress, esp made in the last 6months.  He was on the edge of giving up on me and the project,  I regret so much that he is gone.

 

One thing that was frustrating over these years:  A lot of previous discussion (mostly from technician/audiophiles)  was based on the impossibly simple 1 layer decoder, which didn't meet my goals at all, but being so simple was easier to make it sound good.   Also, the previous single layer decoder was infinitely 'tweaky', and couldn't be used by anyone except me.  Many  were trying to encourage finishing the project as single layer, but given the results today, limiting the goals to single layer would have been so very wrong!!!  I still have the '1 layer' bee in my bonnet.

 

 

 

 

 

 

[John Dyson]

So far, the decoder has had two very very good criticisms.

1)  Vocals sometimes buried

2)  A bit of 'spittle' in some vocal sounds.

 

I'll add a 3rd:

3)  Too much bass (I should have caught it.)

 

ADD-ON:

4)  HF transients too strong (fixed)

 

More criticisms welcome.

 

The buried vocals can sometimes be partially caused by too much bass, but the problem is also about phasing/time delays in the LF EQ.   There just might be a fix, or nearly so ready for a release tomorrow.

 

The 'spittle' problem is now understood, and should be gone in the release tomorrow.   There is/was a problem about what to do with the 6kHz EQ, which apparently needed a different pattern (in V6.0.7B) than for the other frequencies.   By a using yet another modified pattern between 6kHz/9kHz, the 'spittle' problem appears to be removed, and the vocals are generally slightly more clean sounding.

 

There is some testing needed, so the test decodes will be run tonight.   With some luck, those test decodes will represent the release, but most likely there'll be another change needed.  It is no problem if more 'bugs' are found, and will include associated improvements in the next release, whenever it happens.   Goal (like usual) Fri evening.  Never seem to meet the time goal however.   This time, I'll try harder!!!

 

John

 

[John Dyson]

Been reviewing some of the fixes for the recent helpful criticisms, and ran into an interesting issue about one (two) of the fixes.

 

The sibilance 'splatter' problem, often like a 'spittle' thing has been interesting to work on, and tricky to resolve because the design pattern for the EQ doesn't directly afford a fix for the problem.  This mismatch with the regular design pattern has made the sibilance matters to be frustrating to deal with.

 

Another vexing problem is the pre-emphasis/de-emphasis, forever a thorn in my side.   Never have found the 'magic setting', but the closest to correct has been an alternating gain vs. freq like: 6k,-3dB  9k,+3dB 12k,-3dB, etc etc.   Back and forth +-3dB.   Note that another alias that worked similarly, but still not quite correct was a +-1.5dB scheme.   Notice that the average value for each one is 0dB?   It always seemed like schemes that averaged to 0dB tended to work best.

 

The solution for the 'splatter' problem and the 'pre/de-emphasis' might be the same.   First, the pre-emphasis/de-emphasis has a lot of control over how sibilance and other transient-like signals behave.   I just decided to test with it disabled (first time when using a full descrambler.)   WIth the descrambler using *regular* settings and removing the pre/de-emphasis entirely, the sound appears to be better than any previous use of the pre/de-emphasis.

 

Now, the solution APPEARS to be total removal of the pre/de-emphasis and a simple, regular sequence of EQ for the descrambler.  SO far, the results are very favorable..   The tests are not complete, and wont' be complete until the demos are fully run and reviewed, however it would be wonderful to be able to remove the pre/de-emphasis.

 

Thanks much to the person (we know who he is :-)), who, by virtue of the constructive criticism, motivated a very complete still candidate solution to two serious problems!!!

 

John

 

[John Dyson]

The good news it that the pre-emphasis/de-emphasis appears to be no longer needed.  (Attempt to fix the sibilance problem), but the bad news is that because of the moderate structural change in the output EQ sequence for HF (>3kHz), and my propensity to make mistakes, the decoder build today was slightly off-target.   It is good that the error appears to be minor, and any change to bring the decoder with this kind of problem back on target isn't a major or error prone change.

 

It is so close, but the highs are too strong, as if the descrambler was cranked up too high.  (The descrambler acts like a super fast dynamics expander when used too aggressively.)

 

There is a chance for something to come later today, much more likely tomorrow afternoon (+24-30Hrs.)   After the recent, relatively successful release, I am taking it slow for a day or so.

 

It would be plausible that the next release will be functioning just like was expected 3yrs ago...  This project has definitely been afflicted with my own over-optimism!!!

 

[John Dyson]

Been working the last 18/24Hrs, and not found a very satisfactory result for the 's-sibilance' until perhaps now.

Instead of adding on the descrambler as an intimate adjunct, had a wild idea to totally remove the old EQ, even the MF/HF 'join' EQ scheme that has been used for 5yrs.   All of the HF EQ is now controlled by the descrambler, with very interesting (good sounding) initial results.   A few parameters might need to be changed, but for right now the sound is very interesting and intriguing.

 

There is now detail that I couldn't hear before, especially subtle interactions between vocals and the backgrounds.   There is still probably a 'nit' somewhere that needs looking at, but once that is fixed, a 'preliminary quality' set of demos will be done quickly, and make a release.   There is still some review, perhaps moving a block in or out, and the demo-decodes yet to do, but with luck the results/release will be available tonight.   Note that I said 'with luck', and lately luck hasn't always been with us.

 

Again -- this is interesting sounding, and does seem to be 'more correct' than anything before.   You know about the problem with audio though -- sometimes a 'novel' sound is mistaken for 'good' or 'better' sound.   It does seem like a notable improvement has been made.

 

 

[John Dyson]

The new decoder is available, but isn't the 'final' that I had hoped as trivially derived from the previous version.   The change is profound as described in the following, pre created note, and making choices outside of the necessarily modified 'design pattern' can be challenging.   I know that expectations are sometimes lower than I hope, but this release was supposed to be a 'zeroed-in' final, but a major change afforded a VERY VERY major architectural improvement...  

 

There are two versions of snippets -- one is created using '--hfrl', the other without.  WHICH ONE SOUNDS BETTER?   How do you describe why one or the other sounds better?  I REALLY NEED THIS INPUT...  THANKS!!!!   I might be jumping ahead of things by not asking the reviewers first, but this is 'so close that I can taste it!!!'.

 

ALSO:   *THIS IS A MAJOR CHANGE*, so we might have fallen back to too much HF again.   Tell me if there is too much HF as soon as you can.   If there is too much HF, I can estimate how much because of a design pattern (it isn't going to be more than a 'step' of EQ in error.)   With such a change, it just might make the snippet version decision.  I APOLOGIZE FOR THE HISTORICALLY REPEATED TOO MUCH HF PROBLEM...   I understand my hearing problems better, and a bit of kind help can fix the problem in the decoder, hopefully forever now!!!

 

------

 

V6.0.7F is ready, but is half ‘status report’ and half ‘final release’.

 

The decoder is available, but I am very sure that there will be one more release after requested feedback. We need to choose how much HF EQ is needed.

 

There are two snippet demos below – which sounds better? This is NOT meant to be a time waster, but read further below for why help is needed…

 

The ‘s-sibilance’ problem should be seriously mitigated, the buried vocal should also be helped a lot, slightly better bass balance…

 

Very importantly, there are two major internal changes, and these changes contribute to difficulty in deciding the correct sound:

 

1) There is little or no post decoding HF EQ anymore, practically all post decoding HF EQ is done in the descrambler.

2) The descrambler also replaces the need for the pre/de-emphasis.

 

Apparently, previoiusly needing the pre/de-emphasis and the frustrating post decoding HF EQ was forced by a missing HF phase descrambler.

One might suggest that the descrambler is ‘post decoding HF EQ’, but once you see the steps in the ‘EQ’, it is nothing like most of us has seen as ‘EQ’.

 

Snippets with ‘--hfrl’ (the version I prefer, but I am always wrong):

 

https://www.dropbox.com/sh/s52loa0anm4reu8/AAATx8LQATNfFZaaArVoDxSma?dl=0

 

Snippets without ‘--hfrl’:

 

https://www.dropbox.com/sh/g2styl07emrkehs/AABC_Ft92jZRma2oiQXCqGwna?dl=0

 

Usual full public location, V6.0.7F decoder in the decoder subdirectory:

 

https://www.dropbox.com/sh/i6jccfopoi93s05/AAAZYvdR5co3-d1OM7v0BxWja?dl=0

 

 

Explanation below:

 

This release is down to the edge of correctness. There is a layer of EQ that the design pattern nor my hearing can decide. Therefore, there are two versions created by the same decoder. One version uses a specially ‘cobbled together’ ‘--hfrl’ switch, the other is a raw decode. One or the other is closer to correct.

 

Without going into a lot more detail until later, basically – which of the two versions sounds best. Which one ‘needs more’ of something?

 

Instead of making a game out of it, and not saying which version is which, both versions are labeled in the filename, and also the ‘-0’ version of the output is also the ‘--hfrl’ version.  Nothing is hidden.   The additional EQ for --hfrl is -3dB at 24kHz, -3dB at 30kHz, both with Q=0.8409.   Also -0.375dB at 18kHz and at 12kHz/Q=0.577.  The first two remove some grain and creates a more 'tight' HF, the slight rolloff at 12k and 18k softens the effect of the higher freq EQ.   If we decide that --hfrl is needed, I'll do more work on making the EQ clean things up a little better.

 

 

More details:

 

When doing the raw decode, the result seems to be ‘gritty’ to me. That is a ‘tell’ that another rolloff might be needed in the HF. However, in the past when I added a rolloff in this situation, we ended up with a ‘digital’ or ‘too sharp’ sound. Therefore, because there is no design pattern to tell me which is correct, and my hearing is not reliable, I’d like feedback on whether the ‘--hfrl’ (-0) or the ‘other one’ (-1) sounds best.

 

 

THANKS!!!

 

[fas42]

Interesting, John ...

 

Went to Dreamer, that spittle thing seems probably as good as it can get; listened to the RAW, and there's very little in it.

 

Now, with Dreamer the --hfrl came across better than the other, seemed to have more life to it; but the other was closer to the RAW - so, not clear as to the "winner" here.

 

Then tried something new - which turned out to be a good choice - Newton-John, Take Me Home ... here, the --hfrl definitely didn't ring right: the chorus start was too aggressive, and then the drummer's tapping on a high hat(?) sounded like a nervous tick action, didn't blend - the other version balanced nicely, in comparison.

 

If you want more feedback, on another track, give us a yell ...

 

[John Dyson]

3 hours ago, fas42 said:

Interesting, John ...

 

Went to Dreamer, that spittle thing seems probably as good as it can get; listened to the RAW, and there's very little in it.

 

Now, with Dreamer the --hfrl came across better than the other, seemed to have more life to it; but the other was closer to the RAW - so, not clear as to the "winner" here.

 

Then tried something new - which turned out to be a good choice - Newton-John, Take Me Home ... here, the --hfrl definitely didn't ring right: the chorus start was too aggressive, and then the drummer's tapping on a high hat(?) sounded like a nervous tick action, didn't blend - the other version balanced nicely, in comparison.

 

If you want more feedback, on another track, give us a yell ...

 

Thanks again for your input...

I think that with some ideas that you have passed on to me, there might be an improvement or two that can be considered.

 

Also, there was a minor systemic bug that I missed in the new descrambler module, and fixing it should clean up the sound, removing some more 'grit'.   One of the 'balanced frequency' EQs in each descrambler set was offset in frequency, therefore unbalanced the distortion cancellation.   I regret this bug wasting time, but again, your feedback is definitely helpful...

---------------------------------------

 

Your comment about 'take me home' tells me that the highs are still probably a little too hot and the descrambler needs to be tamed a little more.   On one hand, I can hear the signal 'lock-in' on 'take me home', but as you say, it might be too aggressive.  Part of this 'aggressiveness' might even come from the 'bug' mentioned before.

 

The new 'HF phase descrambler' is totally new technology, even to the extent that there are no documents that I can find that show how to use the technique.   On odd artifact of the technique is that HF rolloff doesn't work intuitively because of the 'keyhole' effect.    It appears that more internal descrambler rolloff is needed, yet when it is added, it only affects the most aggressive part of the sound, and leaves the rest relatively unchanged.   The 'internal' descrambler rolloff adjustment doesn't sound like a final HF rolloff adjustment.   It is not a nonlinear design, yet acts nonlinear in some ways.   I am only reverse engineering this sucker, but whoever did the original design had some real magic in their brains.

 

From what I can tell, there is one hell of a lot of energy packed into the 'sparkling sound' from the original FA version.   That 'energy' is what allows the FA sound to be 'bright' and appealing in some ways (but is part of the off-putting effect when I listen to it.)  The descrambler redistributes this 'sparkling sound' energy back into the signal to become a little more like the original, pre-FA recording.   The descrambler design itself comes from intuition, and might still not perfectly match the original encoder.   Gonna reconsider some details, bamd HOPING that the next release will finish up the 'REL5' phase.

 

I knew, that because of the fundamental change, that an additional iteration has been needed.   The beauty for me is that an ambiguous set of building blocks is now gone, and another, less ambiguous but very difficult for me to find the settings is also gone, but the new set of building blocks needs to be refined a little.

 

The decoder design does seem (to me) significantly closer to the minimum (best) design needed, but some of the particulars still need some consideration.

 

[John Dyson]

Gotten some pretty amazing help from someone who isn't using fantastic/fancy speakers or $1500 headphones!!!

Frank, last night, was thinking about the 'roughness' in ONJ "Take me Home Country Roads", and myself never fully satisfied with it.   I could hear the descrambler somewhat 'improve' the results, but then the results became over-emphasized, thereby undoing some of the improvement.   This flaw indicated a bigger problem, but wasn't really obvious until this morning.

Summary about the long blathering note below:   The descrambler is much more finely designed than initially guessed.   By a slight reconfiguraiton of the original 'brute force' descrambler settings to a more subtle configuration, the quality was significantly improved, with less apparent over-expansion, and likely more correct results.   The next release should be very significantly improved.  (Sounds better also :-)).

 

Longer discussion about the above:

 

It would be very tricky to explain how the descrambler works or what it does, but suffice to say that the descrambling is done in 'chunks', each starting at a given frequency, then extending all the way up to the top frequency in the audio.  (inf means infinity) Band 1 is 3kHz->inf, Band 2 is 6kHz->inf, Band 3 is 9kHz->inf, step by step up to 30kHz->inf.   Yes, there are bands above the audible region, but are still important.   Each band also has 'sub bands' whose center frequency is at +221.5Hz and -221.5Hz in addition to the natural inuitive band frequency.

 

The mistake in the previous 'status release' and in all previous tests was that the bands were 'phase balanced' (subject of long research paper), but choosing how many of the 3 subbands in each 'main band'  to use was based on how much expansion is needed.   Because of earlier experiments in creating the pre/de-emphasis it was apparent to me that the 6kHz band  & 12kHz band needed less expansion, so in this new descrambler design 2 of the 3 subbands were disabled for that bands to attain less expansion.   There is an obvious variation in expansion needs, but didn't fully understand what is going on until now.   The eventual inferred design is explained in the subsequent indented paragraph.  Summarizing: there is a phase/magnitude interleaved concept that affords greater/lesser expansion capabilities with a built-in phase distortion 'cancellation' scheme.   This cancellation scheme also helps to define each band as being a pass band instead of a high pass EQ.  (again, research paper needed.)

 

The subband patterns are "basefreq-221.5Hz", "basefreq", "basefreq+221.5Hz", where 'basefreq' is 3k,6k,9k,12k,18k,24k,30k.   I had assumed that 'more' expansion needed all three, 'less' expansion needed just the middle.   Using this pattern based solely on amount of expansion gave plausible, but imperfect results.   Phase had been ignored.   The corrected version, in testing right now, uses a scheme where one phase, with greater expansion, is done at 3kHz, 9kHz, 18kHz, 30kHz.   The other phase, with 1/2 of the expansion is done at 6kHz, 12kHz, 24kHz.   Each 'phase' is implemented as 'greater' uses (basefreq-221.5Hz & basefreq+221.5Hz), while the 'lesser' uses just (basefreq).    This interleaving and balance helps to mitigate intermod problems -- another genius idea in this massively complex audio corrupter/de-corrupter. 

 

Anyway, rather than just looking at magnitude, using & defining 'phasing' in the descrambler tends to mitigate the 'harshness' in 'Take me Home', and generally produces more natural results.   The other 'good news' from these results is that the subtle nature of this 'phasing' idea and being able to specify it in the configuration of the descrambler helps to re-enforce the likelihood of design correctness for the descrambler!!!

 

 

[John Dyson]

Of course, there is another release coming soon, maybe today.

Not gonna rush this next one at all because it is so close to canonically correct, and I don't want to screw up the EQ again.

The next announcement later today or hopefully tomorrow at the latest, will have the HF and descrambler fixes described below.

 

Most notably, the HF will be more tame/correct again.   Sorry about the excessive HF on the 'F' release, but the next release will not have excess HF to the extreme, and might even be a little weak in the HF region.   Hopefully, though, actually correct!!!!

 

====

 

The 'F' release appears to have some of the HF bias that seems to creep in. For the next release, I made a rather drastic decision based on sum of the feedback about 'F'.

 

After the entire set of feedback so far, some corrections have been made to the building block 'slices', and removed any likely unneeded HF emphasis.   The new, experimental result seems to have less excess HF emphasis, at least as I perceive.   On the current set of experimental changes, there is absolutely NO extra HF +dB EQ outside of the simple design pattern for the descrambler.   That is, the descrambler pattern is pure, each chunk identical except for the frequencies in each block, and the only other deviations are for which phase that is selected.

 

Previously, there were 'shims' at 9kHz, 12kHz, 18kHz, .....   All of those 'shims' were removed.   Also, there was some EQ left over from the original HF EQ concept at 3kHz and 6kHz.  

 

With all, every last bit, of legacy design concepts totally removed, the sound seems plausibly more correct.

 

Frustratingly, the non-sine measurement methods applied to the descrambler design produce almost meaningless results.   Since the descrambler IS the HF EQ now, there is currently no usable way to compare the input/output energy at different frequencies above about 2kHz.

 

If the next set of blocks for 'EQ' (descrambling) at/above 3kHz sounds correct to everyone, then there will be a huge likelihood that the HF EQ will be canonically correct,

not just plausibly correct!!!

 

 

[John Dyson]

Just a status update:

Doing lots of back and forth testing, doing comparisions between current and previous versions.   Making sure that nothing is forgotten or new problems don't arise.

 

Some of the previous conclusions were found to be wrong.   The most important is that the HF pre/de-emphasis is still 100% required.  (There is also a still required MF and LF pre/de-emphasis in the design, but need never questioned.)

 

On the other hand, the descrambling is still necessary.   When comparing the EQ-only vs EQ/built-in descramble methods, it does appear that the HFEQ/built-in-descramble method works best.  (Actually, it is the descrambler that does EQ as a side-effect.)

 

The post-decoding HF EQ being effected by the HF descrambler bodes well for the correctness, but also shows the futility of doing the post decoding HF EQ only in terms of 'EQ'.   The architecture of the descrambling method is dissimilar from traditional EQ methods, and trying to separate the the EQ from descrambling creates an 'impossible' challenge.

 

Progress is being made,  and being done very carefully,  The decoder is right on the edge of possible canonical correctness in some areas, but without specs (my continual lament), there are lots of mistakes that can easily be made...

 

Trying to err on the side of caution.   There will be a release in 'days', but not 'hours'.

 

 

 

[John Dyson]

Still working on the descrambler.   It is a little (lot) more intricate than I had originally surmised. Some of these comments are starting to drift into 'audiophile' speak because there is NO engineering way that I know of to easily describe the unravelling of the scrambling scheme.   It is very possible to describe the scheme in the form of matching phases, gains and time delays, but from a 'sound' standpoint, the description seems to 'disorganize' as the actual signal 'organizes'.

 

When reverse engineering the descrambler, it is like a knot made of 7 short cables, where each cable has 3 strands, and  where each cable represents a frequency (e.g. 2.75kHz, 3kHz, 6kHz... etc), and the length is longest for the lowest frequency, and gets shorter as the frequency is higher.   The dimensions of the knot are based upon the frequency offset (+-221.5 and/or 110.75Hz) of the 'strands',  type (1st order, 2nd order), gain and Q of the shelving EQ at each cable strand frequency.  To find the correct pattern to undo the knotted cable, one needs to almost randomly find the correct pattern for the first few frequencies, and then the rest of the pattern starts getting easier and easier as the additional EQ steps are added.    The result of the 'unscrambled' signal is that the transients and ambience tend to 'cohere'. 

 

At this point, there are two post-decoding 'descrambles' that appear to work well.   One version doesn't fully follow the 'rules', but sounds really, really good.   The other approach follows the 'canonical' rules that have been inferred throughout the development.   The next release will *probably* be the version that is slightly at-variance from the 'rules', but sounds 'good', really really 'good'.  (The stereo image on the pure-rule-followed version is too narrow.)   BTW -- the differences between the 'really good' and 'canonical' version are very slight.

 

Starting yet another set of 'demo test decodes', and if they are good enough, depending on how good the results seem, there might be some 'status report' or 'eureka' demos coming in a day or so.   I sure hope that these will be 'eureka', 'the sound', or darned close.  (I sometimes feel like the decoder is like an Oak Island thing.)