'FeralA' decoder -- free-to-use

[John Dyson]

Another follow-up about the 'piano' issue.

 

Since there are so few useful objective measurement possibilities, we need to grab every subjective way of verifying quality as possible.

One of the AS private correspondents brought up the 'not good' sound of the piano on a recent demo.   I agree.

 

The current focus is to use the 'piano' and other instruments for basis of verification or change.   This 'piano' effort is taking longer than I had expected, but great improvements are already available.

As any piano-listening audiophile knows -- the room ambience and micing can really change the sound of a piano, let alone the instrument and artist.   Since the decoder and subject recordings are such a wide and complex set of considerations, it can be overwhelming to match the decoder output vs. expectations.

 

Since the decoder *truly is* right at the final edge of development, esp with the modification of layering (e.g. perfect NR), it is time to do the 'final cleanup' and searching for any possible bug that we can find.

 

Pianos are really troublesome because the question is:  'what should it sound like?'   We know what 'pianos' sound like in general, but what should the recording sound like?

 

I'll be spending perhaps up to 1 wk on this 'piano' matter.   There are recordings where the decoder produces better, seemingly better dynamics.  However, there are also subtle changes in the higher registers where a lot of verification and feedback will be needed.   I am beyond my listening-expertise, and must be more patient than usual.

 

It is good that the issue of the 'botched' sound of the piano came up in a discussion.   Frankly, it'd be nice to have more public discussion, but at least the important issues are eventually coming up.

 

Improvements in sound have already been made, and a well considered 'first try' will be presented in the next day or so.   As mentioned earlier, I'd suspect that this 'piano project' might take a week total.   This is one week that is unexpectedly added to the schedule, but very worth while...

 

I guess everyone already knows -- IMO, just making 'good' or 'plausible' sound isn't 'good enough' for me -- the sound has to be 'plausibly correct', not just 'plausible' 🙂   There are lots of people with more preceptive judgement in areas that I have little.   When doing the massive amount of testing/verification/etc, I can hear something really special appearing.   We are still dithering around 'perfection', but a little farther away from perfection than it should be!!!

 

John

 

[John Dyson]

FA decoder status:

No major changes other than some minor architectural mods.   Goal is to make sure that the result is 'flat' and 'no distortion'.    The distortion from dynamics processing seemed to continue until the previous release.

 

Like usual, I had previously hoped to wrap up the release, and unfortunately the decoding result had previously gotten a valid criticism, so the problem has been very intensively attacked.

The current working result is extremely clean without excessive super HF boost, but I am a little worried about maybe a smidgen too much lower HF (3-4kHz.)   I'll be asking for some quick feedback on it, and if okay and if the bass balance seems okay, then this is 'IT'.

 

I am giving todays work to a friend of mine, a non audiophile person who will be able to listen without audiophile ears.   That feedback coming in the next few days will be very helpful.

 

Assuming no serious troubles, I expect to do a demo upload late tonight or early tomorrow.   I am listening to the candidate right now, and very impressed with the 'easy clarity'.

 

 Will think about it, but an upload of the current version for internal demos, in +12 - +15Hrs really seems like a good idea.

 

John

 

 

[John Dyson]

Status:

Very close to doing the demos release again, but one more day delay...

 

Reason:

Found a very subtle bug, almost imperceptible.   On the encoding operation, the phase is shifted in certain audio bands intending to spread the energy more evenly.   Some time ago, I thought that I found that there should be a 'swap' at [110,440], [440,1.5k], [1.5k, 2.5k] and [4.5k, 6k] to concentrate the energy back together, also getting rid of some of the 'rough' FA sound.

 

After some testing when trying to chase down a sense of 'cancellation' somewhere in the midrange, it appears that the [440,1.5k] shouldn't have been done.   There still might be a need for 'swap' somewhere in the 440/1.5k range, but after some testing, it does appear that removing that range makes a rather nice improvement and a less grainy sound on certain recordings.   After some more testing, it still seems like [440,1.5k] needs to be removed.   Perhaps it might still be a good idea to consider something else in the range, but the improvement is so great on certain recordings, it is probably best to remove the [440,1.5k] range and 'leave it alone' for now.   If a new range is added back in, the testing might delay for another week -- no go.   I want to release this asap, and the improvement is much more than just 'adequate'.

 

The delay is regrettable, but the explanation is essentially complete.   Running some 'short tests' right now, and will offer demos of the 'short test' later on today.

 

John

 

[John Dyson]

The recent delays have been regrettable, but the result is good...

 

It seems like that everytime that I find my own perfection, someone else (a reviewer) finds a 'bug'.   This is okay, but it also causes some fatigue in waiting for the eventual technical conclusion and an excellent decoder.

 

Most recently, I did a casual 'decode' of the 1992/1993 ABBA Gold stuff.   The result was worse than I had hoped, partially because of my own judgement and the fact that I had a hardware failure.   I was depending on inferior technology, therefore some problems were not detected.   Like usual, it was in the middle of a failed experiment that the USB/Headphone interface failed.

 

This most recent set of criticisms resulted in some metaprogramming changes of the descrambler, but with very positive, pretty sounding effect.   The programming is layered, and sometimes overlapped.   There was a different kind of layering that produced subjective superior results.   Alas, measurement is still frustrated where the >3kHz response is even more flat than before, but with little subjective difference because of it.   All substantive changes result from the dynamics processing changes.

 

Part of the search for correctness is driven by looking for the 'smoothest' sound while exposing the most detail.   This goal 'feels' like it would be the most accurate, so that is the focus right now.   'Smooth' doesn't imply lesser dynamics, but instead the dynamics are clean.

 

During the search for approaches to improve the ABBA Gold (and other) results, I looked through the monsterous archives of decoders, finding that the changes over the last year would best be described as the wobbling walk of someone who is intoxicated.   The search originally was intended to be driven by the goal, but unfortunately the guess for the needed EQ/descrambling scheme was wrong.   Then the next scheme was wrong, etc etc until were are probably at the 6th or 7th archtecture, but with no serious architectural changes in the last 6months or so.  Almost all experimental architectural changes have reverted to the current.   The most recent, perhaps 6mos to 1yr test versions now number approx 2100.

 

I am still working on the corrections needed based on the feedback, and once the new interface comes and more testing is done, I'll try some test demos again.

 

This effort, especially in the descrambler has been something like 'bisection' along with 'drunken walk', with some review of history also included.   Luckily, the command structure has been pretty much the same over the last year or so.   This makes comparisons and testing of old versions much easier.

 

The GOOD NEWS -- the best version is the most current.   There might still be some complaints, but the errors appear to be more and more fine.

My goal is 'perfect', and it will have to be nearly perfect and reviewed as such before I'll be able to move forward on other aspects of the decoder.

BAD BAD NEWS:  the descrambler programming has become more complex, therefore creating a speed bottleneck.   The program will currently use fewer cores when decoding, but the effect is to take a little more real time for decoding.   Your computer will likely get less warm and the fans less likely to run at full speed.

 

I don't have the confidence to do any demos, private or otherwise, until I can do some reviews with the new USB/Headphone interface.   My previous one was slightly flawed, and the MB interface sucks -- so looking forward to the new box.

 

John

 

[John Dyson]

Given the most recent (private) disasterous attempt at doing the preliminary demo release, and given I am getting a newer/better headphone interface, I'll be taking the next release more slowly.

 

I keep on getting hung up by my hearing that accomodates overly loud material and material with a bad HF/LF balance.   The problem keeps on multiplying itself until the decoder outputs a caracuture of the recording.   This MUST stop.

 

Recently, got some feedback that some new people might be interested, just might be an artist that has participated in some recordings.   I am not sure yet, and still working on translating that suggestion.   No matter what, I MUST SOMEHOW figure out how to remedy my problem (it is my problem with MY 'wetware'.)

 

Releases much wait until those times that I can reliably A/B.   Soon will be almost begging serious one-shot help from some of the reviewers.

The decoder has practically everything that it needs, except the descrambler metaprogramming.   I am the only person who can do it.   Sometimes, I hope that the descrambler programming can be opened up to a few of the users...   Howeve, command line control is not sufficent, but a control file might.   The problem is that even a control file is too complex to make it practical to program it without deep understanding of the descrambler.

 

Your frustrated friend...

Give me 1-2 more weeks.   I am fully committed to create something that is better than the best of any previous demo.   In fact, I have been referring to some old demos that were reasonably accepted.

 

John

 

[John Dyson]

It has been a relatively long time since giving status...

In the midst of a personal issue, I have also been trying to figure out a persistent bug in the decoder.   The bug is one of those things that pops up only once in a long while, but also is a manifestation of a bug that shouldn't be in the program.  What is this bug, a 'piano problem'?    Sometimes, once in a very long while, the highest registers of the sound of a piano are missing.   Corrections in the last few weeks have significantly, in fact very significantly improved the general sound of pianos, but the result is still not 100% correct.   I know how pianos sound, and previously ignoring the problem did result from how I felt that the piano was miced.  That guess/expectation about the sound being correct was wrong.

 

The problem has been non-trivial to correct, and previous attempts to correct the problem in a more 'brute force' fashion has resulted in incomplete improvement.

Of course, generally 'brute force' 'solutions' are usually the result of desperation, but also in this case actually came from a leftover test.  The 'piano problem' comes from an error where there was too much expansion in the DA layers.   Of course, 'DA' processing is almost fixed, only 'adjustable' by certain historical modifications utilized for vocal enhancement along with other things.

 

In fact, the most recent and only significant  complaint has been about the sound of 'pianos'.   Up until now, after a few minor undisclosed corrections, the sound of pianos has been correct with a frustrating exception.   Fixing the 'piano bugs' without a full correction is just as bad as leaving the complete set of previous 'piano bugs.'   Any such bugs also leaves the possibility of other problems that haven't been detected yet.

 

Premature releases have happened because I didn't know that there had been a problem.   Since this piano issue is now known and mostly understood, it would be totally wrong to ignore it and allow any of the problem persist.

 

This is an incredibly difficult project, and there has been massive progress in the last few months.   We are zeroing in.  I am hoping that we are close enough to the target to claim that the program is totally complete.   It might have been easy to leave the bugs as when the I had previously declared success a few weeks ago.

 

I deeply regret about previously being overly confident about the decoder technology being complete.  There is no good answer other than it already sounds 'okay' *again*.  Mostly sounds good in the current test version.   The decoder sounds suprisingly good, except for a single slight and very subtle problem (seldom only noticeable on piano material.)   SOUNDING GOOD ISN"T GOOD ENOUGH THOUGH.

 

Of course, the goal is that the decoder have no 'sound', but should only reproduce the recording before the final stage of processing.   Apparently, that goal has not quite been met.   Maybe the next demo will be good enough to publically announce?  I sure hope so, but I am expecting to make the upcoming version available as long as it is better than the previous  internal test.   The next set of demos that are plausibly good will be distributed.

(There are personal matters that have really slowed down progress -- the matters are being resolved in a very positive way, but still some distractions.)

 

John

[John Dyson]

Good Good Good news.....

Summary of the message below -- problems solved.

After all of the steps of learning and the incremental improvements, the decoder IS working and now appears to truly be 'decoding'.

Demos coming in 'days' not even 1wk, possibly this weekend.   Decoder Linux binaries will come along with demos, Windows approx 1wk later.

 

-----------------------------------------------------------------------------------------------------

 

Details:

 

The descrambler programming had been driving me to the edge of insanity, but think that it is now much better understood.

First, it has been very clear for a long time that a descrambler is needed, but the term 'dispersive audio processing' isn't well documented in literature, and that is exactly what the descrambler is.

Second, I have had an incorrect picture in my mind as to the processing/metaprogramming needed.

Third, I have been too arrogantly impulsive to think this through -- honestly, I have had significant experience in multiple fields with significant EE/DSP knowledge -- arrogance had gotten in my way, and erroneously applying old knowledge rather than learning what the turkey (descrambler) really does.   Over the last few weeks, especially with other important personal matters, I have had a LOT of time to think from scratch.

 

The descrambler is like a 2.5 dimensional graphic equalizer WITH TEETH.   Of course, that is not a technical definition, but shows my emotional sense of it's behavior.

 

Most previous versions of 'metaprogramming' had the expansion turned way-up, to the point where the descrambler couldn't do very much.   Instead of 'turning it way up', it is now set towards with a bias in the negative gain direction in the midrange.    Previously, the midrange was forcing the LF & HF to work way too hard, maxing out the processing capability.   Now, the midrange is set in the 'negative bias' direction.    In the neutral state, the descrambler has a negative dynamic midrange gain (it is still essentially flat on average.)   The difference now is that the midrange now appears only when needed, and the other bands can operate properly.

 

I know that without a lot of context, the indented paragraph is gobbledygook, but basically -- the gain is more centered so the descrambler can work more effectively and CORRECTLY, yet not sound intrusive.   With careful, careful review -- I am reasonably confident that the decoder is 'decoding'.

 

John

 

[John Dyson]

The project has gone full circle, where I first noticed the patterns in recordings in about 2011 when listening to ABBA Gold.    This now shows a very close to correct decoded version (snippets) of ABBA Gold.

 

Until now, cleanly decoding ABBA Gold has been problematic since the recordings are incredibly intense, possibly because the recordings might just be 'FA'* encoded master tapes, true mixed masters not just distribution masters.   Now below are some snippet demos for a reasonably well decoded copy of the 1992 ABBA Gold.   More ABBA Gold is just as good....   As always, there might still be some minor changes needed, but very minor.  The difficult, very challenging part of the project is complete.   Of course, the PROGRAM is not complete, just the research and algorithms are essentially correct.   There is still 'normal programming' yet to be done.

 

It would be inappropriate for me to comment on the results of the decoding operation...  Judge for yourself.

 

Regrettably, I don't have the 'raw' CD snippets ready, but will create the snippets before tomorrow.  If there are 'nits' in the recording, it might be best to refer to the raw CD versions to compare -- I WILL make the references available soon.   This has been a big 12yr effort, so I'll be resting a few extra hours to make up for lost sleep.... 🙃   (The decoder programming project didn't really take 12yrs, but the program itself started in approx 2017.  The progress and programming was like a drunken walk.)    I approached the program as a hobby instead of a true professional project, and that was a PROFOUND and almost FATAL mistake.

 

https://www.dropbox.com/scl/fo/893ai6e9jc5udeeux7ahb/h?rlkey=s7bhlrt88wp6ajh2hgy35mil2&dl=0

Given the very special nature of these recordings, the Dropbox online player is especially heinous -- please download first!!!

 

 

*FA:  a new name needs to be chosen, since success is near, if not fully achieved.

 

I haven't recently chatted with online reviewers/friends...   Been VERY distracted, but the forced time away had encouraged more 'thinking' rather than 'doing'.

Will re-establish in about the tomorrow timeframe.

 

John

 

[John Dyson]

Follow-up:

The examples are almost the final version, but I am still trying to decide some minor issues.   For example, there is a subtle detail vs brightness tradeoff.   The change would be slight, and like in a few other ' very close' settings, it is still important to try to be 'correct' rather than 'sound good'.

 

There are definitely cases where the sound could seem superficially better, but actually be less correct and sound worse when carefully listening.   The FA process is now well understood, but there are very fine adjustments, especially WRT the 'pilot' processing.    The 'pilot' makes a monsterously huge difference, and ignoring the pilot makes the decoding result thin and 'just wrong'.

 

The decoder is at the point that large scale changes will no longer be needed, the sound is generally as it will be!!!

 

John

 

[John Dyson]

When doing additional reviews and getting feedback, I have a few comments:

 

1)  There might still be a minor correction in the LF, it might need to be a +3.0 dB instead of 1.5dB.   I cannot tell with my hearing, but either setting is plausible. *

2)  I am still trying to figure out a way to attain higher detail (without harshness.)

 

*  With my totally unstable and unreliable hearing, I must think in terms of plausibility and probability instead of hard core obvious A/B answers.   The current unknowns, currently only these two, will easily be resolved as the two last variables.   No matter if my hearing isn't always working well, it still sounds VERY INCREDIBLY GOOD now.

 

The matter of the LF might be most obvious and notable.   The needed change in LF (if needed) cannot really be done by simple EQ, but either +1.511dB (the current) or +3.022dB follow within a narrow set of rules of correctness.   At one time or other, either setting gives *incredible* clarity.   *IMPORTANT*  this +1.5 vs +3.0 dB difference is for ABBA GOLD ONLY.   The decoder is okay for other recordings.  

 

The decoder does NOT increase clarity, but stabilizes the signal so that it appears to have more detail, that is demodulates the smear.

 

With any luck, and the final decision about the LF choice, I fully expect a full demo release for ABBA Gold (snippets for public versions)  before the end of the weekend. 

 

John

 

[John Dyson]

Well, with the feedback given and my hearing returning to some degree, some very minor changes in settings were needed for best quality.

You might have noticed a little bit of 'grain' in the sound, which comes from minor errors in the programming.   This 'grain' exists in the descrambler, whether or not it is corrected on the output.   However, the correct settings are when this grain is cancelled.

There is a massive number of A/B comparisons with massive numbers of 'rabbit holes' (erroneous sets of choices) that have been traversed.    We are now in the correct trajectory, with minor perturbations that I couldn't detect.   The sound is smoothed out, but will be spending into the early weekend to be sure that all of the nits (little buggy 'tells') have been corrected.

 

Also, along with the 'unevenness', some of the expansion/compression was blunted off, so some of the dynamics were weaker than should be.

 

All in all, I might have been 2days premature on the ABBA Gold demos, but all is very good.   ABBA Gold, being close to true master material, is also very difficult for the decoder.   Where the normal Polar/Polydor copy of 'SuperTrouper' might sound good though the decoder, decoding that simple material hides lots of sins.   ABBA Gold is a nightmare to process, and it seems like that ANY error results in some audible error.

 

It is only the last 2wks that the decoder produced a nearly 'plausible' result on ABBA Gold, but it was so much better on the first early demo, and now sounds even 'clean' or nearly so on the current test version that I am playing with.

 

PLEASE wish me luck to avoid mistakes when making the next set of demos.    My hearing is JUST BARELY returning to a good state, and should be able to avoid any serious problems on the results (upcoming demo.)   It is possible that I still made a mistake on the upcoming demo, but the errors are more and more vanishingly small.   (Small errors can produce very unpleasant results though -- the decoder REALLY DOES need 0.0005dB accuracy or better, or weird distortions start appearing.)

 

John

 

[John Dyson]

Again, I regret the delays...

It seems like 'wack a mole' had appeared again, once a new minor (truly minor) bug was corrected, then another important bug appeared.   The new derivative probably was a new error in the L/R<=>M/S conversion.   For a long time, it appeared that for lowest distortion and best image, the factor should be sqrt(sqrt(2)) wider.   Then, more recently it appeared that it should be 1.33, then most recently 1:1.    These results were always obtained by massive A/B testing, so these weren't specious -- except....

It seems like the descrambler (or lack of) was skewing the results for the DA layer...

 

I can provide numerous other little stories like this, but the bottom line is that the sound is more and more like the FA original, but more clean.    I am happier and happier with the normal Polar (and early Polar) Mamma Mia on the ABBA S/T album.    ABBA Gold is now really good, and other material like Simon & Garfunkel and various orchestral recordings are turning out pretty good with great imaging based on A/B comparisons.

 

This project will not be stopped until the results are as correct as reasonably feasable.   The current results are closer than ever before, even though BEFORE I didn't realize how far off (in error) that the earlier versions were.

 

There are still some trusty reviewers handing me my a** at times, and I haven't even been asking all of the available reviewers to help -- because until at least a few are telling me 'good enough', then I won't waste the time of everyone else.   HOWEVER, historically a few had said 'okay' from time to time, but there had always been vetoes.   I truly think that it is getting better.

 

Also, there is a new ABBA reviewer who might give a different viewpoint than before -- so, we have 3-4 very interested ABBA-fan reviewers, that is including me.    If all of us end up agreeing, this will be a 'good day'.

On the other hand, almost all normal consumer digital distributions must ALSO appear to be correct, so this is not just an ABBA thing, but orchestral/classical is just as important or moreso!!!

 

The next message with either be in several days or when something *really good* happens.

 

John

[John Dyson]

Current set of demos available.

I am interested in if the highs seem close to correct!?!??!

These should be technically vastly improved -- but it is still very helpful to have bugs pointed out.

 

This is the package of 95 recordings/snippets and A/B/A/B comparisons.

There are now 10 second, 20 second and 35 second comparisons.   20 seconds added because 35 sometimes seems too long.

 

Demos:

https://www.dropbox.com/scl/fo/7poa9d1a9nzbia41ukze7/h?rlkey=04jzqxqze4vftg0rv14hpt4x9&dl=0

 

Remember that the online Dropbox player is of generally lower quality than the decoder output.   It is best to download the examples first, but a quick, fuzzy review can be done directly online in the Dropbox player, if you want.

 

 

John

 

 

[John Dyson]

6 hours ago, John Dyson said:

Current set of demos available.

I am interested in if the highs seem close to correct!?!??!

These should be technically vastly improved -- but it is still very helpful to have bugs pointed out.

 

This is the package of 95 recordings/snippets and A/B/A/B comparisons.

There are now 10 second, 20 second and 35 second comparisons.   20 seconds added because 35 sometimes seems too long.

 

Demos:

https://www.dropbox.com/scl/fo/7poa9d1a9nzbia41ukze7/h?rlkey=04jzqxqze4vftg0rv14hpt4x9&dl=0

 

Remember that the online Dropbox player is of generally lower quality than the decoder output.   It is best to download the examples first, but a quick, fuzzy review can be done directly online in the Dropbox player, if you want.

 

 

John

 

 

The reviewer feedback so far is better than it has ever been.*   The demos are not fully complete, but the standard 95 tracks is ready as noted in the previous posting.

*  Very early on, there were some misguided positive reports on the old single layer decoder -- but those results were one-off, and not repeatable at all.   THE GOOD DECODING RESULTS NOW ARE EASY TO REPEAT!!!!

 

 

So many 'breakthroughs' have been found through the years, and there have been many, many steps of obscure techniques needed to make the decoder truly perform a decoding operation.   There have also been NUMEROUS cases where early decisions that seemed to produce optimum results were found to be wrong.   The recent (hopefully final) corrections were partially YET ANOTHER L/R<=>M/S conversion change.    After all of the various choices, the scale factor for width ended up being 1:1 through the entire system.   Apparently, the earlier evaluations were distorted by the descrambler behavior.

 

Given the consensus so far, there won't need to be any more structural/algorithmic changes.   IF there are programming changes, they will be simplification intended to improve readability.   Perhaps, maybe some speedups if possible.

 

Several weeks ago, there was a VERY pleasant find....   On many older recordings (e.g. Take 5, older POP, etc), the hiss is OBLITERATED.   Earlier I had expected some NR, but until about 1month ago, there was disappointment -- both myself, and an interested user.    NOW found out about 1month ago, the NR is working at 100%....   There is almost no audible hiss on the old Brubeck recording, I mean the very slightest, faintest hiss modulation -- most people might not even notice it.   This hiss improvement alone might make the decoder worthwhile to use!!!!

 

NO matter, assuming that the consensus stands, there will be NO changes other than for porting purposes until the Windows binary is working.

For the first time in 7yrs, I feel just a little relaxed.

 

------------------------------

I do have an idea for the future, perhaps something that just might fit into a DAW plugin -- a descrambler-only 'partial decoder' might be practical.   Some of the amazing NR will not work without the DA layers, but signal clarification just might work.

 

There will be more demos tomorrow or next day.

 

John

 

[John Dyson]

Added ABBA snippets (ABBAsnip subdir.)   Unfortunately, 'TheVisitors' needs EQ, I might add a corrected version later today.

 

https://www.dropbox.com/scl/fo/7poa9d1a9nzbia41ukze7/h?rlkey=04jzqxqze4vftg0rv14hpt4x9&dl=0

 

Even though the 'decoder' does a really good job of 'decoding', sometimes recordings need EQ because of 'mastering' before FA encoding.   This 'mastering' done before FA encoding might sometimes help an album sound better as FA encoded material.

 

For example, this EQ/mastering had been done on the 1992 ABBA Gold, 1993 More ABBA Gold and the standard Polar 'The Visitors' albums.   So far, I haven't noticed before-FA mastering on other albums, but as I might find them, will make note and mention the fact online.

 

The EQ isn't very complex, but I didn't do the needed EQ after FA decoding because of my poor judgement on these matters.   It will probably take significant amounts of 'TLC' to complete a good-quality and possibly accurate job of undoing this 'mastering'.

 

John

 

[John Dyson]

On this most recent demo, essentially complete, we did get some critical feedback on the sound of pianos.   The flaw had been real and since at least partially mitigated.

 

In the next, actual release, the sound of pianos should be significantly improved, plus decreasing a slight odd hollowness in the upper midrange that might sometimes be irritating.  Both problems, the 'holllowness' and not-so-good piano sound come from the same source.

 

I didn't notice the problem until it was brought to my attention last night.   After understanding the problem to be real, I found an error that certainly contributed to the problem.

A side benefit to this correction is that the hiss modulation of old recordings (e.g. Brubeck) was slightly improved.    When compared with earlier versions, the hiss on older recordings was already vastly improved.   This new  'piano improvement' makes yet another small improvement to the hiss modulation.

 

The sound of pianos might not be totally correct yet, but it really does seem to be significantly improved.

No matter what, it is unlikely that a new complete set of demos will be needed before the release.

 

There will probably be a demo of the very good noise reduction for Brubeck, a Carpenters recording and 'The Girl From Ipanema'.   It all depends on how well the additional testing goes later tonight.

 

John

 

[John Dyson]

Starting a new set of snippet demos, hoping to start the re-porting of the decoder to Windows this weekend.

 

Important note:  ABBA isn't the sole source for tests/demos.    I typically run tests/comparisons from orchestral, Jazz, Pop everything but Hip Hop only because I haven't accomodated to that material.   ABBA is a 'guilty pleasure' and hearing defects while listening to an ABBA recording encouraged the genesis of the project.   For the project, different genres help to contribute test materials because each genre tends to have different characteristics, and SADLY I have no spec to design the decoder.   This thing MUST be tested/A-B compared over and over again with absolute dedication.

 

When doing the demos/doing quick checks/etc, it is POSSIBLE that it might force a V20T-RELTRY3, but I promise will be a very minor change.   The pilot might still need a different setting for handling peak expansion.   Incorrect pilot handling can cause garble, and I am a bit worried about it.   There is LITERALLY one value that can change/decrease the problem, if any -- just that it is one of the 'last details', just added a day or so ago!!!

 

V20T-RELTRY2:

https://www.dropbox.com/scl/fo/2zs1lpozq7wqsicbgqsbb/h?rlkey=rbuc5zqsycmyv86bom1rjcj8h&dl=0

 

ABBA is available now...

95 demos (mix of pop and orchestral)...   probably in 1/2 day or so.

Carpenters snippets...   probably in another 1/2 day....

 

So far, we have the ABBA tree complete with the Hits and Hits2 from Polar also (very high quality.)

I had to 'demaster' the Hits albums, and TheVisitors still needs some 'tender loving care'.   The 'Hits' albums don't have the standard sound, and once worked on just a bit more, should be astoundingly good.   Right now, they seem to be pretty good.    The normal 'The Visitors' seems to have some EQ, and it takes quite a bit of time to clean up EQ (de-master).   I am already 100% occupied while working on the decoder itself, so 'tweaks' for ABBA must wait for a week or so.

 

The rest of the ABBA albums seem to be unadulterated before FA encoding, that is, they are the 'standard' releases with the 'standard' kind of sound.

 

The decoder processing of the 'pilot' signal is further advanced, and perhaps over utilized the pilot on this version?   Don't worry, the bug is slight, if it exists.  I was having trouble choosing how much to let the pilot take control of the signal processing, and maybe made the pilot processing too strong?

 

Later on tonight/tomorrow, I plan to upload the standard 95 recordings, etc.    After this demo, whether or not there are criticisms of V20T-RELTRY2, I must re-do the Windows port ASAP so that people can give the decoder a test-run.   There are individuals who are interested, and I cannot string them along ANYMORE!!!

 

Sincerely!!!

John

 

[John Dyson]

When producing the latest (not announced yet) version, I found out how to better use the 'pilot', which now allows suppressing more of the hiss pumping on old recordings.

Unfortunately, I might have screwed it up.

This version is NOT announced, because I think that I screwed up some aspects of response balance when improving the distortion/making the pilot work better.

I am offering this RELEASE0 version ONLY to keep from holding back, but it is probably not the correct result becuase of a last minute upgrade and bad A/B choice...

 

https://www.dropbox.com/scl/fo/wh971wlto6cshxr7xa1ui/h?rlkey=1jor4aviv81hb1sp8onpi17w7&dl=0

 

Last minute changes are often deadly, but a bug was DEFINITELY corrected -- just that another 'wack a mole' popped up...

 

SORRY -- gotta try again.

The good news is that we have a good, maximally detailed sound -- just that I screwed up one of the controls on the pilot.

 

John

 

[John Dyson]

There was a last minute add-on that carefully and without damage should gouge out the hiss and place certain low level details back into their correct level.   Unfortunately, I just learned of this mechanism associated with the pilot 2-3 days ago -- or more accurately, just learned how the pilot scheme works at more detailed level.

The 'pilot' is a cascading kind of residual left over from the original FA encoding.   The original concept for decoding using the pilot as one of the components was tricky, but incomplete.

 

On the finishing touches of the release due to today, I found the missing aspect of the pilot processing.   The 'cleanup' was started after a fairly darned good set of private reviews.   I know that one of the reviewers is getting a little impatient and irritated because of the delays and seeming disregard of the positive opinions.   I do not disregard the positive feedback, but I promised myself that the decoder do 'AS GOOD AS POSSIBLE'.

 

There aren't many years before the acuity of my hearing will become fuzzy enough that continuing on the project will be even more difficult.   Therefore, I feel that I must do AS WELL AS POSSIBLE as EARLY AS POSSIBLE.    If I delay this final 'zeroing in', then the goal of 'as good as possible' will not be met.

 

There is only ONE of ME available for the project, even though there are some kind people who are really helping.

I must delay this release until I can figure out the complete set of details for the settings on the pilot control.

 

What does this new mechanism implement?    A last bit of super high speed expansion that hides noise pumping on the very old recordings like 'Dave Brubeck' Take 5.   The recently 'reasonbly good' reviewed decoder (a previous version of V20T) still had noise pumping on, for example, 'Three to Get Ready' on Time Out, and also the Carpenters 'Bachrach-David medley'.   There are other recordings, including orchestrals, where the noise pumping is improved with the more complete set of pilot capabilities...

 

I know that I keep asking this, but please give this another week.   I believe that with the complete set of pilot processing, the result will be as close as possible to being correct.   Hardware can NOT decode the FA signal without sounding muddy -- this software is NOT a mirror image of the FA encoding scheme, but is much more complex.   This complexity and challenge is no excuse for my mismanagement, because my fault of premature announcement has been more damaging than the technical mistakes.

 

The promise has been 'AS GOOD AS POSSIBLE', so I must delay for approx one more week.   I might be testing the 'working pilot' version right now, but cannot assume that my judgement is accurate.   I need help from people who are 12 time zone away, along with 5 time zones away, and sometimes in my own time zon,  and each person is available on a hobby basis.   No matter how hard I push myself, it will still take approx 1wk to make sure that the pilot is working correctly.

 

With regrets, but still hopeful...

John

 

[John Dyson]

Here is a recording, lots of hiss, showing significant improvement.

This is where the decoder is as of right now (time of posting.)   However, LOTS more testing and need more critical feedback from reviewers AFTER more refinement.

 

With these demos (SRC -- BEFORE, V20T58 -- DECODED), it is still possible to hear some hiss pumping.   Also, it seems like there might be a bit of a lower midrange issue, perhaps on the dB level of error.   Still not sure, the settings have a relatively coarse grain, so the final is a matter of additional careful A/B comparison exercises.

 

The decrease in hiss is VERY profound, but still the pumping needs some investigation...   I AM a perfectionist, and ANYTHING that can be improved MUST be improved.   If it cannot be improved, then we are close to success...

 

BEFORE (converted from 176.4k/24bit distributed, commercial source):

04-Three To Get Ready-SRC.flac

 

DECODED (converted from 88.2k/FP decoder output):

04-Three To Get Ready-V20T58.flac

 

As you can hear -- there is progress, and is frustratingly close to 'working'.   The results must be good on more recordings than just these two, and the results are VERY VERY close to success...

 

Sincerely,

John

 

 

[John Dyson]

A potentially interesting feature will be in the decoder -- enabling/disabling the descrambler.

 

The difference when enabling/disabling the descrambler  is 'interesting', and clarifies some of the damage done by the ubiquitious (almost everywhere) compression process (I just call it misnamed 'FA'.)   Note that doing an A/B comparison of enabling/disabling the descrambler will show that gain control can happen WITHOUT an explict variable gain/variable resistance element...   The technique is almost 'Star Trek'.

 

Earlier on, I had really thought that the FA process was just a few layers of DolbyA type processing, but that was only part of it.   The DolbyA style processing is very important for proper recovery, but the descrambler is key.   In fact, the descrambler produces an 'odd' signal that the DA layers utilize.   It 'sort of' super-drives the DA layers.

 

All together, I think that anyone listening to the demos will be able to hear the incredibly profound noise reduction -- MUCH MORESO THAN ANY OTHER DEMO.   It sounds more than just DolbyA alone, and does seem to be audibly on the order of 15-20dB.   An FFT scheme is likely to produce more NR artifacts with less signal recovery for the higher levels.

 

Pretty (very) amazing...

 

John

 

[John Dyson]

The delay/regroup for completing the decoder appears to have been a very very good idea!!!

 

When doing one of the dozens of reviews of the sound and source code BOTH, I found a rather nice bugfix for the discriminators.  This might be more important as a 'bugfix' instead of as an improvement.

 

The change is to make the discriminators into a push-pull configuration instead of a parallel configuration.   It might seem like push-pull might be an obvious choice given an EE experience and history.   However, the discriminators are non-trivial when seeing the whole picture of the descrambler design.   When doing the initial work, thinking about 'push pull' vs. 'parallel' would only confuse the already complex set of design choices.

 

Frankly, I haven't been totally comfortable with the sharpness/edginess of the sound.   The 'edginess' can seem like detail, but really isn't.   High order odd distortion components can sound something like 'detail', if the timing and structure of the odd components.   The acceptability of the decoding results probably comes from limited recent experience hearing truly high quality material.   The 'FA' scheme has definitely distorted my own sense of 'good quality' vs. 'mediocre quality'.  Much of the 'edginess' in the decoder output comes from partially correcting the signal, but not cancelling out the FA encoded distortion products.   In a way, because of other changes, the distortion products pass-through the descrambler rather than being cancelled out.

 

Hopefully, the new version, with the updated discriminators will help acceptance to some degree.   Since the structure of the descrambler hasn't changed at all, and only a component has changed, this modification will not affect any release time.

 

I have delayed the project over and over again, but many reviewers have been conditional in their acceptance of the decoding results.  The conditional/limited approval has made me a little unhappy, but should have strongly motivated a proper correction to the 'nebulous' problem.  Certain changes have been ignored because of stronger focus on the structure.   Frustratingly, it is only the last month when the structure of the decoder is 'practically perfect'.    The components, (e.g. pilot, discriminator, necessary EQs, 6 DA layers) are still possibly 'not practically perfect'.

 

The goal in this phase is NOT to focus on code improvements, but bugs must be addressed.   IF this new discriminator is adopted, it will be as a bugfix and not an enhancement.

 

Working in a more relaxed way, but still with the focus/intent...

John

 

[John Dyson]

Good news:  inroads in quality...

Bad news:  there are lots of little nits that are best mitigated now...

More minor good news:  the decoder is becoming so good that the anti-fog can run more quickly (the decoder is mostly faster.)

 

A few days ago, I just did a demo for one of the reviewers -- about the incremental improvement through a certain set of changes in the decoder.

If I am not happy about a release in a few days, I'll make this just mentioned demo available -- the demo DOES show real progress beyond the FA original.    I'll be doing my very best to make a release available in a few days.  *I am PLANNING to make the release available middle/early this weekend, but my plans are not usually fulfilled;  I am so tired making my bad time estimates, so all I should claim is that THE DECODER IS REALLY GETTING BETTER...   It has generally been *good* in the last month or two, but again NEAR PERFECT is the only acceptable result.

 

I am focusing very  intensively on some very subtle little nits.   Most are related to cleaning up vocals and backgrounds.   These errors happen because of very minor mistakes when choosing a setting.   On most material, the difference between 'correct' and 'incorrect' isn't worth worrying about.

 

Since I did notice that the decoder is becoming so very close to 'canonically correct', these small adjustements are 'low hanging fruit' in some ways.

All of the 'adjustments' are either a +-3dB setting being more or extended to lower freqs.   Once in a while, there are true tweaks, but the number of new 'true, less than 3dB tweaks' appear to be diminishing.

 

True challenges right now:

*Getting rid of the slight amounts of 'grain' in vocals.   The FA original is usually the worst, most grainy.   Properly adjusted decoder tends to be most clean of all choices.   No matter what, the decoded version has been more clean than the original.  Near perfect is the only valid result.

 

The other challenge is when making changes in HF, especially that affect the 2kHz to 9kHz range, sometimes add a bit of a 'bump' in a bad spot somewhere in the 3-6kHz range.   These bumps are true bugs, but with variably bad hearing are sometimes a challenge for me to detect.   *MOST FREQ RESPONSE SETTINGS ARE NOW SOLID, but sometimes the dynamics changes can make it seem like the freq response is different.

 

John

 

[John Dyson]

Just had a PM friend reach out about the decrease in frequency of postings.

Nothing is wrong -- just tedious stuff is going on.

 

All is good, just trying a new way of working.   The decoder is maddeningly close to deadon practical perfection.   Some of the settings require great accuracy, and errors in two settings, even something so simple, can be difficult to center back onto the correct settings.  The general sound of 'Accuracy' is becoming more and more obivious -- even though the result is wobbling a little, I am becoming much more comfortable.

 

The decoding results are 'wobbling around' with the numerous test versions being tried.   I'd suspect that anything that would be uploaded just this minute might be 'the best ever', but the decoder needs to be 'better than the best ever'.

 

The 'wobbling around' is in an audibly narrow range.

Full runs of decoding most of my active archives are being done, then I go through and spot check on specific recordings along with doing random listening exercises.

 

Believe me -- never worry about the project disappearing because of frustration -- I just don't get very frustrated!!!

More than likely, the postings have slowed down since there is little new to report other than the humdrum/grind of searching for 'better' settings.

 

I do owe the interested parties a more consistent update...

I might have something good in about 3days, so there will be a message, 'status report version of demos'  or real semi-final demos (again.)

 

During this 'vacant' time, at least 2 full runs of the demos are done per day.  If there was a demo version that is amazingly version, I'd post it.   At this point, we are wobbling around very good results.

 

John

 

[John Dyson]

Since it appears that I had 'disappeared' too long, it seemed to be a good idea to produce a 'status report' set of demos.

This is the 'V20V-RELEASE8' set of demos, intended for internal use.   I have been making 'fake internal releases' every 1-2 days and then carefully listening for flaws.   During the 1-2 days, I have avoided listening to ANY recordings, trying to avoid any accomodation.   This method appears to be successful, but only helpful when there are very few variables that need manipulating (LIKE NOW.)

 

Here are the current V20V-RELEASE8 demos.   The directories with names like 'fscmp10' mean that there is an A/B/A/B cycle of 10 seconds.   The first of the sequence is always the 'DECODED' version.    'Flacsnippet' provides the entire (necessarily limited sized) snippet.

https://www.dropbox.com/sh/i6jccfopoi93s05/AAAZYvdR5co3-d1OM7v0BxWja?dl=0

 

These demos are so close to perfect that it was difficult to do this demo release and not wait a few more days.

The known flaw is that there is a disturbance in the 4.5kHz to 7.5kHz range, which is a missing phase swap.   It has been difficult to decide on whether or not the phase swap is needed.    Even semi-trivial 'binary' subjective decisions can be erroneous, and it appears that the RELEASE8 version of the 4.5k to 7.5k phase swap was wrong.

RELEASE9 will be notibly more smooth sounding.

 

Also, I have mentioned the profound amount of noise reduction.   If you listen to 'Kathy's Waltz', 'Take 5' or 'The Girl from Ipanema', the noise reduction, with minimal pumping, should be noticeable.  MUCH IMPROVED over previous demos.    Sadly, for 'RELEASE8', I forgot the most egregious example of hiss:  'Three to Get Readiy', but will be in the RELEASE9 demos.

 

Regrettable, a true release isn't ready yet, but I am doing the best QC possible.   Since 'almost correct' hasn't seemed to be good enough, the decoder will be made as correct as possible.   I do believe that the architecture IS correct, so 'decoder correctness' is a matter of finding the correct parameters.

 

John