DolbyA decoding feedback -- 'feral' examples (yes/no)

[fas42]

Some more good news ... I tried the Carpenters' Please Mr Postman example, and the processed version definitely comes across better! The sense of some congestion found in the before sound was reduced, and subjectively there was better definition of the various elements, in the after - good one!

 

Nat King Coles' Walkin' My Baby Back Home brought an interesting, disputed aspect in strong focus ... height! Yes, height - the orchestral on the left was well above the position of his voice - and the before was particularly strong in the sense of this - but, the after had lost this to a great degree, they were almost level now ... so what is this telling me? I'm not sure - I could live with the sound of either of these versions, they're fine by me.

[rando]

"I cannot imagine that I will ever buy software and individually "white glove" re-process my own CD's.  So whilst I am both fascinated and very impressed with the work you are doing with Dolby A, I am not sure how this can ever be of benefit to others"

 

The above comment by @Confused got stranded in a recently locked subject. 

 

Not only am I interested what @John Dyson has to offer by way of reply.  On at least a few occasions I've envisioned the scenario refuted above being strongly sought after as one piece on the road to heightening enjoyment of this hobby.  Where improvement comes from breaking the inertia of static recordings through personalization of owned works.  An idea certainly less taboo these days with nearly every living musician possessing a classic rock album in their past reissuing it in a matured form.  Much less taking it back on the road for fans at least as familiar with the works (read flaws) as themselves after years of repeated listens.

 

 

 

Apologies if I'm off topic.  8 pages of searching failed to turn up another Dolby-A specific thread.  

[John Dyson]

The DolbyA decoding issue is a real problem.  Proper mastering/processing/preparation is the domain of the distributors.  However, they have been very lax in the last approx 35yrs (since the original CDs.)  Even some of the early CDs (e.g. 99 LuftBallons/99RedBalloons) are not decoded -- in fact that also has the CD pre-emphasis.  Many, many of the digital distributions created from material recorded between the 1960s through 1980s is NOT decoded.  Frankly, IMO they generally suck, and is one reason why I quit my 'hifi' listening back in the late 1980s'.  I couldn't stand the terrible sound -- I thought it to be 'impure' to have to apply EQ for material to be listenable.

 

Back in the early 1980s, I had purchased the Sheffield Labs direct-to-disk album (vinyl) with 'Thelma Houston, I got the music in me', and loved it.  I thought -- my new CD will sound better!!! -- NOT!!!.   I suspect that was the last CD that I purchased in the 1980s or so.  That CD sounded TERRIBLE.   Just recently, a nice friend in the forum was nice enough to lend me a digital copy (from the CD) of 'I got the music in me' for testing -- lo and behold, it was DolbyA encoded!!!   THAT is the reason why that CD sucked, but the vinyl was so nice!!??!?!   Lack of proper mastering on digital material WAS what did me in back in the 1980s or so!!!  WOW!!!  By the way, my decoded copy of 'I have the music in me' is BEAUTIFUL.  THIS IS A 100% true story!!!

 

Should the audiophile do their own decoding?  I don't know -- it is probably not best for most people...   It isn't terribly difficult to 'decode' material, but the problem is: "How do I undo the damage done by the errsatz mastering?".   If I documented the various processing necessary to get to the point of decoding, I could come up with a list of - maybe - 10-15 sox commands, maybe 20-30 command sequences.  'Re-processing' already released material is tedious at the least.  The other problem with decoding your own material -- without tones -- is to find the calibration level.  So, there is the problem of finding the proper calibration AND finding the corrective EQ (perhaps two filters, usually at 3kHz and maybe 9kHz), and determining the filter parameters.  There are MANY variables to reconcile -- if someone wants the hobby of decoding material -- go for it...  That 'hobby' (necessary to support the decoder project) has almost turned me into a crazy person.

 

I am NOT frustrated by most people in the forums not taking me up on an offer of a free decoder to use for a few months -- I only did that as a matter of politeness and avoiding the idea that I am keeping something away from everyone.   My frustration is that I had motivation to write the decoder, because SO MUCH material that I liked was not properly mastered.  (Secondarily, starting 4yrs ago, I didn't realize how damned difficult an approx 60 transistor circuit would be to emulate.)

 

So -- the above is my thought process.  I have NO resentment against audiophiles who believe (somewhat rightfully so) that it would be distracting to deal with decoding the material for themselves.  The problem for all of us -- the distributors who seem not to care about the quality of their product...   Then a generation and 1/2 of audiophiles have been purchasing poorly made product -- that IS my frustration, and I really want to help!!!

 

John

[rando]

If it was easy nobody would be engaged enough to actually do it.  Your labors will pay dividends. Especially for the next generation and a half. 

 

I think your work will have complementary benefits only a few will instantly appreciate.

[John Dyson]

I thank you for the sentiments.  The worrisome thing (as a legacy issue) is that there are only two places in the world who have the source code, and a DolbyA decoder will never be written again.   If it was written in a large company, it would have cost approx $500K to write or not be done at all.  (Of course, my specs and the performance goals were much higher than a commercial product would be -- no reason to do the decoder if standards werent very high.)

 

Here are the problems with the project now:  1)  I had a dearth of listeners/testers, as I am not sufficient and my project partner has even more hearing problems than I do.  2) We had absolutely no support for software reviews 3) We had absolutely no support for proper testing of any kind.  4) Many of the publically 'released' versions had significant bugs.

 

All of the 'bug' problems are really solved -- I just removed the last two hacks in the code (there were two hacks intended to work around a behavior problem that has since been corrected.)   Unlike a true DolbyA -- the DHNRDS DA can line-up complex sounds from old recordings, and fully avoid the 'dithering' kind of sound and the decoding fuzz associated with a DolbyA decoding.  The software FINALLY meets those goals entirely.  I am afraid that too many people have seen the older versions -- but I needed criticism for testing/verification.  Also a lot of demo material has necessarily been already damaged commercial recordings.

 

Regarding the source code repository -- the DA decoder is truly a $500K development project if in the 'real world'.  There is new (patentable if I wanted to) technology in the decoder, and this intellectual property has to be managed correctly.  I need the energy/wherewithall to at least disclose some of the new techniques (they go far far beyond the Orban distortion avoidance patents), and even to properly internally document the source code.   The two copies of the software - in the hands of Richard Hess and myself, is fully dependent on two 'very gray hairs' and I am worried that it will disappear somewhere into the ether.

 

With the very slight amount of interest being shown (yea, a few recording engineers and maybe an recording archive site or two), there is not enough mass to be able to maintain the software and keep it in the public/recording world awareness.   I promise, the project and the software WILL die given the very terrible state of the recording distribution industry and the acceptance of really bad quality recordings - even from those spending multiple $10k on their equipment.  (Garbage in, garbage out, but the Emperor's new clothes stll apply.)

 

*  My own guess as to why the 'loudness wars' hit the high fidelity audio world stems from the DolbyA leaks and the totally cr*ppy sound of the CDs for the pre 1990 material.  That terrible quality decreased the standards of the listeners -- even the 'elite' listeners, also frustrating some of them into thinknig that 'all they need is better equipment'.   Of course, the problem has NOT been in the equipment as much as it is the horrid quality of the recordings.  Then, with the distorted standards, and the love for loudness for attention -- NOTHING pushed back on the loudness wars.   This lowering of standards started:  NOT DECODING DOLBYA IN THE DIGITAL BEGINNING.

 

John

[lucretius]

Is the second (processed) version of Please Mr. Postman a little bit less compressed?

[John Dyson]

9 minutes ago, lucretius said:

Is the second (processed) version of Please Mr. Postman a little bit less compressed?

It sounds significantly different -- actually it is a third generation beyond previous demos.  I'll run the carpenters albums starting now, should be done in a few hours.  I'll put up a few demos.  Believe me -- the original material (before decoding, sold as consumer product!!!)  is unlistenable (at least to me!!!)

1) there was a bug that I couldn't see...  2) I had workarounds in the code (the last two 'hacks' left in the code), 3) once I found the bug, removed the bug -- then subsequently removed the 'hacks' were forced by the bug.

 

As of now, the code is canonically provable.  No hacks at all.  I can now prove the code (at least the decoding math) for every component in a DolbyA unit.

 

Demos will be made available by about 7:00PM EDT USA time 4sep (I hope :-)).

 

John

[John Dyson]

2 hours ago, John Dyson said:

It sounds significantly different -- actually it is a third generation beyond previous demos.  I'll run the carpenters albums starting now, should be done in a few hours.  I'll put up a few demos.  Believe me -- the original material (before decoding, sold as consumer product!!!)  is unlistenable (at least to me!!!)

1) there was a bug that I couldn't see...  2) I had workarounds in the code (the last two 'hacks' left in the code), 3) once I found the bug, removed the bug -- then subsequently removed the 'hacks' were forced by the bug.

 

As of now, the code is canonically provable.  No hacks at all.  I can now prove the code (at least the decoding math) for every component in a DolbyA unit.

 

Demos will be made available by about 7:00PM EDT USA time 4sep (I hope :-)).

 

John

The DolbyA decoding (or DHNRDS DA decoding) demos are in the dropbox below.  I included .mp3 for convenience, but if you hear any swirls -- try the flac, the 'swirls' or 'phasing' won't be in the flac.

 

Caveat1: I 'decoded' the recordings from released 'consumer' material.  I did my best to 'undo' the ersatz mastering, and then applied the DHNRDS DA decoder to the 'prospective' DolbyA encoded recording.

Caveat2: I suck at 'mastering'.  If there IS a mistake to be made, I'll make it and embarass myself. We are NOT demoing my (lack of) mastering abilities :-).

 

The 'undecoded' material is 'RAWundecoded', and the 'decoded' material is 'V0.9.7K' - the version of the DHNRDS DA decoder used.

I'd suspect that with a true master tape, the results would be better -- usually with actual master tapes, the results are pristine.  However, when one starts with ersatz mastered material, and NO calibration tones, the effort can be daunting.

 

Each of the three examples comes from different albums, and I didn't re-adjust the calibration for each album (might have required minor changes), but it appears that the correction for ersatz mastering is the same for each album.  The 1969 and 1970 albums do seem to require a different correction, but these come from 1972 for 'Top of the World', 1975 for 'Please Mr Postman', and 1976 for 'Sing'.

The '13' version of Please Mr Postman is from the remastered singles available from HDtracks and elsewhere.

* There is a HUGE difference in the 13 version of Postman and the 04 version -- they ARE different.  'Postman' probably did need a remix, but '13' has too much compression (IMO.)

 

Here is the dropbox:

https://www.dropbox.com/sh/ciwfhzbhod3brnn/AABF1_QE7AVBOj0EGFe4KT5Ga?dl=0

 

[John Dyson]

I added some equivalent 'olvia newton john' snippets.  the undecoded compression is pretty obvious there also.

 

https://www.dropbox.com/sh/d2dyuonx8fk3pwv/AAArjQgYoo_Skpy8YSiOuxpha?dl=0

 

[fas42]

One of the solutions in the future - for those who don't want to tackle the underlying problem, which is that playback flaws accentuate characteristics in the recorded sound that make it unpleasant to listen to; leading in turn to the "only the best recordings are worth listening to!" syndrome - is that full unmixing, separating of the musical strands of recordings will be developed as a software solution - this is partially in place right now.

 

Will be cheap, readily available, largely automated to massage any recording to one's own preferences - the mastering engineer's job will go the way of the farrier's ... .

[lucretius]

1 hour ago, John Dyson said:

I added some equivalent 'olvia newton john' snippets.  the undecoded compression is pretty obvious there also.

 

https://www.dropbox.com/sh/d2dyuonx8fk3pwv/AAArjQgYoo_Skpy8YSiOuxpha?dl=0

 

 

I listened to the files and here's what I found:

 

Karen Carpenter

#1, #2 and #4 -- I thought I heard a difference but too close for me to call it.

#13  There was one point after she paused singing where some instruments kicked in -- the unprocessed file had less definition of those instruments than the processed file.

 

Olivia Newton John

#4  - I don't think I heard a difference.

#14 and #19 - the processed file gets rid of the some fuzz

#23 -- the processed file has less "digital glare"

 

Does this make any sense?

[sandyk]

45 minutes ago, lucretius said:

Olivia Newton John

#4  - I don't think I heard a difference.

HI Lucretius 

 Just for one example .

 I find that the decoded version of "ONJ-Take Me Home Country Roads" has the chorus sounding much cleaner (less harsh sounding) and clearly defined. When her voice kicks in it is a little softer and purer sounding , with noticeably improved separation between elements for the rest of the track. This version sees me wanting to hear the whole track.

(Admittedly though, I have had quite a bit of practice listening to John's various examples)

 Regards

Alex

[lucretius]

14 minutes ago, sandyk said:

HI Lucretius 

 Just for one example .

 I find that the decoded version of "ONJ-Take Me Home Country Roads" has the chorus sounding much cleaner (less harsh sounding) and clearly defined. When her voice kicks in it is a little softer and purer sounding , with noticeably improved separation between elements for the rest of the track. This version sees me wanting to hear the whole track.

(Admittedly though, I have had quite a bit of practice listening to John's various examples)

 Regards

Alex

 

I heard those differences initially but I thought that the two tracks were not level matched - the processed track appeared more quiet. When I turned up the volume on the processed track vis-a-vis the unprocessed track, the two sounded very similar to me.

 

[sandyk]

24 minutes ago, lucretius said:

 

I heard those differences initially but I thought that the two tracks were not level matched - the processed track appeared more quiet. When I turned up the volume on the processed track vis-a-vis the unprocessed track, the two sounded very similar to me.

 

 Yes, the original is a little louder, however I can still hear the clear improvement with the decoded version when increasing the listening level.

Alex

[John Dyson]

There are two differences (but we probably don't hear hiss like I used to hear) -- there is definitely less hiss (look at the spectogram -- the whole purpose for DolbyA - not originally intended as a sound-defect.)  Next, the excess ambiance should be suppressed (that compressed sound, we are all used to hearing nowadays, is artificial.)   IMO it is a travesty that EVEN audiophiles are generally used to the damaged, compressed, especially unnatural sound qualities of undecoded material.   Our hearing accepts training for sound-defects very readily.  (Such training has been a major difficulity in my testing -- one reason why near the end of the project I had reached out for kind people like Sandyk, who might be more discerning than myself.)

 

Difference not caused by DA decoding: the high level transients are not changed much -- all 'action' is at -20dB or below (at MF frequencies) or a bit higher at the higher frequency bands.  It is the HF density damage (the unnatural dynamics) that are morphed back into 'more natural' dynamics.   You would only measure small increases in SOX crest factor, even though the peak-RMS ratio can sometimes be very significantly improved (1 dB improvement in peak-RMS is a very substantial quality improvement.)

 

It was that unnatural (but NOW tolerated ) compressed sound that turned me off back in the late 1980s -- the hiss didn't bother me all that much -- I definitely preferred hiss over DBX on tape...  I turned away from the HiFi hobby back then because the 'solution' to media quality (CD) ended up being worse than a well-maintained disk -- mastering issues.  One more thing -- when listening to the undecoded material, actually there has been EQ to suppress the HF boost from the compression.  My hearing has become trained to detect and know when to add small amounts of EQ in the 9kHz range -- the damage to such EQ is one reason for certain kinds of damage to sibilance.  (the 6kHz range is the most vulnerable region for damaged sibilance, and what is often enhanced in one way or another -- a great frustration when trying to DA decode such additionally damaged material -- basically ending up in the dustbin if I cannot correct that mess.)

 

We really need to get rid of the acceptance nowadays of that tolerance for HF compression.  It doesn't sound natural.  Of course no recording sounds totally natural, unnaturaly, HF specific compression is yet another defect that is artificially created.

 

Bottom line -- the difference in these recordings is probably much greater than the difference between a good $200 amplifier and a good $10k amplfier.   These improvements (generally -- not always, sometimes leaving in the compression DOES help) are much greater than moving from good quality electronics into esoterica.

 

The frustrating thing for me -- is that the audiophile should ALREADY be naturally purchasing the properly processed material -- yet will still spend 10X as much money for an amplifier that looks like a Christmas tree with single-color lights, yet IF there is a sound improvment it is 1% as much an improvement as properly mastered material.  The decoding mistake had started in the middle 1980s, and has created a culture that has morphed into a contributor to the 'loudness wars'.

 

I respect the individual perception of each person, but once one gets used to 'less damaged' material, then the damage & improvements become obvious.  Up until now, my patience was tested because I kept on improving the DA decoder, but it is truly 'perfect' now (as perfect as DolbyA CAN be decode -- which is still imperfect), and I am finally doing my *permanent* decoding effort (just started) for my library.  Otherwise, the contents of the library will continue to distract me for the (nasty) HF compression so common nowadays.

 

John

[John Dyson]

2 hours ago, lucretius said:

 

I heard those differences initially but I thought that the two tracks were not level matched - the processed track appeared more quiet. When I turned up the volume on the processed track vis-a-vis the unprocessed track, the two sounded very similar to me.

 

I DID not peak level match the Olivia material -- it is now matched.  The Carpenters was already level matched.  But even IF one does match the peak levels, then the average decoded levels will tend to be less.  This is because the dynamics are cleaned up and the various signal components are restored to their proper levels.   DolbyA compression tends to INCREASE the loudness (to a lesser extent than the extremely aggressive compressors nowadays), and DolbyA decoding TENDS to DECREASE the loudness given the same signal level.

 

BTW -- I did NOT compare before and after decodes -- just checked to make sure that the 'afters' weren't insanely bad.

 

John

[fas42]

22 hours ago, sandyk said:

HI Lucretius 

 Just for one example .

 I find that the decoded version of "ONJ-Take Me Home Country Roads" has the chorus sounding much cleaner (less harsh sounding) and clearly defined. When her voice kicks in it is a little softer and purer sounding , with noticeably improved separation between elements for the rest of the track. This version sees me wanting to hear the whole track.

(Admittedly though, I have had quite a bit of practice listening to John's various examples)

 Regards

Alex

 

Alex, I'm pleased that you pointed to that track - but not for the reasons you've stated, . You see, the undecoded version is the one with the sparkle, with the life energy in it; the decoded one is somewhat flat, squashed - like material has been placed over the tweeter - it's the original I want to keep listening to, .

 

The powerful choral intro is exactly the type of musical burst which is an excellent test of a system's competence - does it get it right, or does it mess it up? If a rig delivers that intensity at 100% clean, then it's magic to listen to ...

[sandyk]

19 minutes ago, fas42 said:

You see, the undecoded version is the one with the sparkle, with the life energy in it;

 

Frank

It would appear that your preferences are for a lot of added distortion, and unwanted Dolby A artifacts due to not being properly mastered for the CD release. Your system also appears to be quite lacking in the area of HF detail, or you wouldn't feel the need for all this HF crud and distortion that wasn't part of the original recording session.

Were you listening from a Laptop by any chance, instead of your main Audio system ?

[Ralf11]

he just hasn't finished tuning the system yet

[fas42]

12 minutes ago, sandyk said:

 

Frank

It would appear that your preferences are for a lot of added distortion, and unwanted Dolby A artifacts due to not being properly mastered for the CD release. Your system also appears to be quite lacking in the area of HF detail, or you wouldn't feel the need for all this HF crud and distortion that wasn't part of the original recording session.

Were you listening from a Laptop by any chance, instead of your main Audio system ?

 

Alex, I'm afraid the shoe is on the other foot - HF crud is what nearly all high end rigs are so excellent at producing; I've hear enough over the years to last lifetimes ... it takes the cleaning up of what ails the particular setup to finally eliminate the ugliness that most audio systems are so prone to adding.

 

You see, what you end up are the qualities that are so enchanting when you hear live music making - or haven't you been in the presence of a choir going for it, recently?

[pkane2001]

23 hours ago, sandyk said:

HI Lucretius 

 Just for one example .

 I find that the decoded version of "ONJ-Take Me Home Country Roads" has the chorus sounding much cleaner (less harsh sounding) and clearly defined. When her voice kicks in it is a little softer and purer sounding , with noticeably improved separation between elements for the rest of the track. This version sees me wanting to hear the whole track.

(Admittedly though, I have had quite a bit of practice listening to John's various examples)

 Regards

Alex

 

There appear to be a couple of notch filters, in addition to a slightly lower level above 8.7kHz and a couple of unexpected 2dB dips at 8.9k and 9.8k:

 

Here's the same plot with the undecoded file on top:

[sandyk]

39 minutes ago, pkane2001 said:

 

There appear to be a couple of notch filters, in addition to a slightly lower level above 8.7kHz and a couple of unexpected 2dB dips at 8.9k and 9.8k:

 

Here's the same plot with the undecoded file on top:

 

You would need to discuss this with John.

[John Dyson]

2 hours ago, pkane2001 said:

 

There appear to be a couple of notch filters, in addition to a slightly lower level above 8.7kHz and a couple of unexpected 2dB dips at 8.9k and 9.8k:

 

Here's the same plot with the undecoded file on top:

 

[John Dyson]

Those 'notch' filters are essentially an inaudible error (I made a mistake in a sequence of filters in the demastering correction in sox.)  I skipped a few 500Hz bands -- I can hear ZERO difference in that kind of case.   Many times, my quick manual runs have run the correct demastering sequence -- depending if manual or not, and depending on what album is being decoded.   Except for the notches, any 'dips' are corrections against ersatz mastering.  (Attempted correction by the distributor to do EQ making compressed DolbyA material sound tolerable.)   I thought that I had fixed the filter sequence -- but there is a different set of filters for each group. 

 

It is a NIGHTMARE to deal with feral DolbyA material.  I try hard, but my mind/focus isn't on it -- and make mistakes every time.  Also, I maintain at least one script per album, a set of correction functions for each group (usage dependent on the album.)  Then, when I am testing with the material from one group, often I forgot about a loose end when going back to another one.

 

I have some more explanations for any other (no-notch) bumps in the response, but it is basically demastering (correction of ersatz mastering.)

 

On the before and after, where there is a diminished level at low levels -- that is to be expected.   The higher levels are maintained.   The lower levels are subject to the downward expansion in the DolbyA decoding.

 

I'll do a rerun without the filter mistakes -- you'll hear ZERO differences (but the mistake is NOT a good thing anyway.)   I suck at mastering -- I make major mistakes EVERY TIME I atttempt.  My mind is usually on the wrong thing -- thinking about what is happening in the decoder, instead of actually using it.

 

I am currently doing a massive, multi-album test decode of another group, but will do the 'Take me home' again to show that it is easy to fix.  I don't think that ANYONE will hear a difference (but my mistake is truly a mistake.)   It will be tomorrow morning before I can produce 'Take me home' again, and a few hours to do all 48 cuts at the highest quality level.   (The EQ error was before decoding.)

 

Give me access to any true master tape (with tones) in your possession (I'll sign an NDA), and I can show how pristine the decoder works when starting with properly maintained/unprocessed/undistorted material.  I cannot share my own copies of proprietary materials (even 10seconds worth.)  I made a personal guarantee - but it is very difficult to find material with tones in the public arena.

 

John

 

[John Dyson]

4 hours ago, pkane2001 said:

 

There appear to be a couple of notch filters, in addition to a slightly lower level above 8.7kHz and a couple of unexpected 2dB dips at 8.9k and 9.8k:

 

Here's the same plot with the undecoded file on top:

 

 

Okay -- the demos with the 'K1' and 'K1A' suffixes do not have the bumps that you are talking about.  Also, I spent more time in setting the calibration level -- anyone listening should get the sense that every last bit of detail on the recording is being reproduced (in the flac version.)   All of the errors were in my sox script and the calibration setting, not in the decoder.

 

In the K1* versions, I applied no sibilance correction (there is often a bump in the 9kHz range which is artificial sibilance -- in the leaked DolbyA material.)   I don't know if the 9kHz bump is intentional, or it results from enhancement causing a 'friction' distortion between the HF0/HF1 bands in the DolbyA HW encoding.  (The HF0/HF1 bands actively interact in the 9kHz range -- the dance must be precise.)

 

The calibration change brings out a lot more detail also.

 

The detail in the results are closer to what a master tape will provide, but still the leaked DolbyA consumer stuff is not 100% trustworthy, so sometimes there is a little excessive 'love' added.   Significant phase errors can be troublesome even though the DHNRDS uses phase insenitive detectors, still too much EQ done by the 'mastering' can cause phase shifts with untoward effects.

 

Listen to 'please mr please' or 'Let me be there' -- very closely.  Methinks that every last bit of detail in the recording (modulo the 16 bits and 48k) is reproduced there.  (There is a certain 'sweet' sound that happens when the DA decoder is set correctly.)  When the HF0/HF1 dance lines up perfectly -- the sound is astonishingly good.  (It is sometimes tricky to get that precision on feral material.)

 

https://www.dropbox.com/sh/d2dyuonx8fk3pwv/AAArjQgYoo_Skpy8YSiOuxpha?dl=0

 

 

 

 

John