Jump to content
IGNORED

'FeralA' decoder -- free-to-use


Recommended Posts

Update on the *great* quality of the decoding examples  (just reviewed a few -- perhaps a 10% suboptimum rate in the list that I created, but still darned good.)

 

After all of the decoding tests that I have done in the last year, there has been a checkered past on quality.  These latest results are astoundingly good.  I don't know if many of you remember my travails on Supertramp recordings, Fleetwood Mac, Carpenters, ABBA, etc.   Going back to the previous results and comparing with the examples given yesterday -- these new results are, for all practical purposes, using the correct sources, near perfect or *perfect*.

 

SIMPLE decoding results, better than my previous, but not as good as my 'tweakd' examples yesterday can be achieved with JUST --fcd or --fce, and proper setting of --tone=.   It is REALLY that simple now.   No arguments are really needed...   For example, the 'precise' results for certain recordings are --fcd=G, but in reality, very good results can be gotten with --fce alone.   There is a very slight difference in the 9-11kHz region between --fcd=G and --fce, the difference is dB or so.   Just --fcd or --fce are good enough probably for most casual listening (except for VERY FEW select recordings and certain good classical stuff.)   The less accurate setting might actually sound better -- but accuracy is my goal now.

 

Today is documentation update day.  I think that I am going to be forcibly distracted away tomorrow also, but gonna try to update all of the background information and start integrating more work from others.  It might take a few more days as I am much worse at communcation skills than doing 'new' development.

 

It takes me several times longer to do concise documentation than it does with someone with normal writing skills.  So, be patient.  I am going to TRY to stay away from programming for a few days.

 

John

 

 

 

Link to comment

Hey -- while trying to figure out how to describe choosing between the modes -- I found a REALLY CLEAN recording, and think that it might be interesting to show the comparisons (both simple objective measurements and the subjective-oriented results.)   This weekend and probably a few more following days will be usage guides and docs.   The decoder MUST be easy to use and usability has been the primary focus for the recent updates.   The problem with the decoder design is that I have NO objective measurement for general performance, and tweak-tweak-tweak is so very time consuming, error prone and frustrating!!!!

 

The recording for testing/checking decoding is from the 'Brothers In Arms' Dire Straits album, selection#2, Walk of Life.   I have two relatively supurb commercial versions, and the decoded version to demo.   I also have a 'j random' version from an old CD -- only one cut from it, but is much more compressed than the two commercial versions shown here.

 

* You can really hear the relative lack of compression in the decoded version.  Also, the raw CD has more HF compression, but even the MFSL version is somewhat compressed.  (As mentioned previously -- I have another CD, SOMEWHERE that is much more compressed than any of these.)

 

The decoding parameters were: "--fce --tone=-13.45 --xpp=max --wof=1.19", for the decoded version.   There was ZERO post-decoding EQ.   A full copy of the album would be improved by the following, but I did *not* use this on this demo: "--pe45=2,0.75 --pe375=4,0.75 --pe1k=4,0.75  --pe9k=K,-0.75 --pe10p5k=4,-0.75 --pe12k=2,-0.75".   This EQ makes the tonality sound closer to the MFSL version, but the general differences in sound character are still intact.

 

* when decoding, I found a very significant decrease in fog when using the --xpp=max instead of --xpp alone.

 

There is a difficulty in choosing between "--fce" and "--fcd=G", because they are so similar, but different enough for getting the very best, most precise results.  (--fce is 1 [email protected], stop at 22kHz, --fcd=G is [email protected], stop at 10.5kHz, and 1 pole@9kHz, stop at 20kHz.)   You can see where the results would be similar, except the overlap on the --fcd=G, and some recordings DO benefit from this.

My challenge is that I am trying to describe how to choose between --fcd=G and --fce in documentation, and I need to learn what the differences sound like.  Gonna try to explain it.

 

Here are the snippets: start at 60seconds into '02 - Walk of life", lasting for 30 seconds.   There is ZERO monkey business in this demo, except a serious attempt to present with approx the same apparent audio levels (e.g. stripping the playback level metadata, audibly matching the levels are reasonably as I can.)

 

Also, included is a screen grab of the primitive SoX stats, with the summary

 

1)  MFSL version. (pk - RMS: 16.37)

2)  Raw, high quality FA CD.  (pk - RMS: 16.63)

3)  decode of the high quality FA CD. (pk - RMS: 17.82

(Actual snippets at the end).

 

Quote

 

[jdyson@localhost demo]$ for fn in *{MFSL,ORIG,decoded}*.flac; do echo "${fn}":;sox "${fn}" -n stats; done

Dire-Walk-MFSLCD.flac:

Overall Left Right

DC offset 0.000237 0.000115 0.000237

Min level -0.347260 -0.344482 -0.347260

Max level 0.311829 0.311829 0.307526

Pk lev dB -9.19 -9.26 -9.19

RMS lev dB -25.56 -25.56 -25.56

RMS Pk dB -20.04 -20.57 -20.04

RMS Tr dB -96.50 -96.39 -96.50

Crest factor - 6.54 6.59

Flat factor 0.00 0.00 0.00

 

 

Dire-Walk-ORIGCD.flac:

Overall Left Right

DC offset -0.002250 -0.002137 -0.002250

Min level -0.335266 -0.335266 -0.326965

Max level 0.298340 0.279999 0.298340

Pk lev dB -9.49 -9.49 -9.71

RMS lev dB -26.12 -26.23 -26.01

RMS Pk dB -20.52 -21.55 -20.52

RMS Tr dB -96.41 -96.33 -96.41

Crest factor - 6.87 6.53

Flat factor 0.00 0.00 0.00

 

Dire-Walk-decoded.flac:

Overall Left Right

DC offset 0.000008 0.000008 -0.000003

Min level -0.384827 -0.384827 -0.319122

Max level 0.323273 0.319244 0.323273

Pk lev dB -8.29 -8.29 -9.81

RMS lev dB -26.11 -26.26 -25.96

RMS Pk dB -20.67 -21.42 -20.67

RMS Tr dB -96.72 -96.72 -96.47

Crest factor - 7.91 6.42

Flat factor 0.00 0.00 0.00

 

[jdyson@localhost demo]$ echo -n 'MFSL Peak-RMS:';calc '25.56-9.19';echo -n 'ORIG Peak-RMS:';calc '26.12-9.49';echo -n '

Peak-RMS:';calc '26.12-9.49';echo -n 'decoded Peak-RMS:';calc '26.11-8.29'

MFSL Peak-RMS: 16.37

ORIG Peak-RMS: 16.63

decoded Peak-RMS: 17.82

 

 

 

John

 

 

 

Dire-Walk-MFSLCD.flac Dire-Walk-ORIGCD.flac Dire-Walk-decoded.flac

Link to comment

I am still working on a beginning user guide, it is focused on decoding procedures and steps.  One difficult thing for me, I am trying to avoid technical blather -- a 'beginner' isn't going to want to be overloaded with even more complexity and details.

 

Also, I ran into an INTERESTING, very INTERESTING case.   I will NOT be covering this case in the beginners guide, but will definitely mention the possibility...

 

The Al Stewart recording, 1976 Year of the Cat, CD FA 3253, is a REALLY interesting case.  It has a defect that make decoding challenging, and also requires TWO PASSES to decode to non-FA.  It is a strange beast.  Here are the data points:

 

1)  Defect: The CD has the lows rolled off.   To properly decode, the following EQ is needed:  bass +3.0dB@80Hz/Q=0.8409 and +3.0@20Hz/Q=0.8409.   Without the LF correction, the material produces all kinds of LF distortion -- ugly.  (Yes, you have to add bass to make it clean -- it is NOT an overload kind of problem.)   DolbyA decoding does require a nearly flat response (small perturbations are okay, but not multiple dB errors.)  This kind of thing is one of the 'tells' that prove DolbyA compression in the recording.  Decoding and encoding must be mirror image at the lower frequencies.

2)  The first pass must have the HF1 decoder turned off (--hf1off), --fwide=none (no image modification)

3)  The second pass must have the MF and HF1 decoders turned off (--mfoff, --hf1off), --fwide=tpop (the minor stereo correction)

 

There are several other odd settings that are used for the recording.   Some day, when I can really start contributing to @lucretius's list, there will be a full decoding spec for the CD.   The recordings are sounding very good now, no grain, but still not totally satisfied -- the decoded CD sounds as good as any commercial version that I have heard -- and MUCH better than the non-decoded CD.   Using two passes is a bit troublesome, but since the DHNRDS has very low distortion, it isn't as bad as if trying to use DolbyA HW.

 

The caveat about recordings like this:   the anti-distortion code might have strange interactions if used two times in a row.  Part of what the code does is to remove parts of the signal that look like modulation products  (it is a complex thing as it doesn't just remove modulation products, but does so as it sees what it is doing -- really, really subtle, implicit design.)   It does seem that the anti-MD does still work well with two sequential passes, even in the higher quality, anti-MD modes.

 

Note about the reason why they are using DolbyA HW units for multi-band compression - it is probably about exquisitely expert design  in the attack/release circuitry, where even though it still produces distortion, it is much less than a simple, straightforward design.  It is also very optimal -- you might notice that most compressors might have more simplistic attack/release circuitry, but the DolbyA is highly dynamic and very good.  I simply don't think that attack/release can be any faster, producing low distortion, that is, without the crazy code in the DHNRDS.  (On the DHNRDS, I have tested the anti-distrotion  by turning off the attack/release -- creating effectively about 2msec attack/release, and producing very low distortion.  It can only do this and still sound good because of the anti-IMD and anti-MD code.)  Hardware from the '60s couldn't do what the DHNRDS does.

 

This (CD FA 3253) is a  perfect example of the FA encoding being intentional, and using the DolbyA HW units as selectable mode multiband compressors.   By turning off the HF1 compressor when producing the material, then the result has less distortion.  The HF1 band is complex and has extra & strange time delays causing interactions into the HF0 band.   It is a reasonable decision to turn off the HF1 compressor if using a DolbyA as a multiband general purpose compressor.  (Also, the extra 5dB of compression above 9kHz is kind of silly and is problematical anyway.)

 

Why they are using DolbyA HW units for multi-band compression? - it is probably about the brilliant attack/release circuitry, where even though it still produces distortion, it is much less than a simple, straightforward design.  It is also very optimal -- you might notice that most compressors might have more simplistic attack/release circuitry, but the DolbyA has highly dynamic characteristics and still very good.  I am impressed every time that I review the design concepts.   Amazingly, he made a compatible unit both with diode (linear gain vs. current, exponential gain vs. voltage), and jFET (which has extreme variability from device to device.)

 

More important question:  Why are they doing the compression AT ALL?  I think that my posting would be bleeped if I write MY opinion about using the 'bad-touch' compression.   Mindless compression on  recordings distributed on contemporary digital media (90+dB dynamic range) is 'bad touch'.  Attempting to recover from the damage, is 'good touch' :-).

 

 

John

Link to comment

New release:  V1.4.7C.   Very nice, very important.  The best sound yet.  Super-duper clean.  When I do A/B, the previous version wasn't bad, but this is REALLY REALLY smooth.   You an sometimes hear where the midrange might be suppressed on the older version, or something is partially missing.  These problems are greatly improved.

 

In the midst of working on documentation, had HW failure, needed a full re-install on Windows box, etc.   Well, back running and able to do the release.

 

No substantive user differences, the best decoding commands are --fcd, --fce, --fcf, --fcd=G, --fce=G, --fcf=G.  Same as before.  There have been some very substantive sound improvements, but normally use exactly the same settings as before.   The decoding is much more robust, but once in a while, you might benefit from perhaps one '-'.  If you still have scripts that used two '-', that is okay, but normally one '-' is all you really need.

 

There were some bugfixes (e.g. disabling HF1 works more correctly now, but when do you really need that?)  There also are some minor frequency tweaks that zero in on the correct EQ more precisely -- the result being better cancellation of generated artifacts.  Most of the time, you won't notice the improvements, but they are really helpful.  There are some other internal improvements, but nothing that means much to the end user.

 

The decoder is MUCH MUCH more tight WRT the proper EQ.  The good news is that the raw decoding of DolbyA material is perfect, but the EQ has been a painful and blind reverse engineering.  Here is an example, of the VERY TOUGH two level decoding as done against the Al Stewart recordings.  I am not claiming perfection, but the original CD is little and narrow sounding, and the decoded version is much more normal.   This was decoded by:  sourceCD -> decode0 -> decode2 -> output snippet.   The entire album is done very well like this example.  Unlike a single level decode, this one is two levels and much more difficult to do.  IT REALLY DOESN"T SOUND BAD, esp when compared to the compressed original.  If I heard a snippet of the DolbyA master, it would be a little better.  However, this is very listenable:  (Al Stewart, Year of the Cat, If it doesn't come naturally).   This recording is tougher than 'Dreamworld' to do (which, btw, is near perfect now.)   Dreamworld is also two layers.  * I tried to balance the levels -- the CDorig version has lots of compression and gives a 4dB level advantage.  I tried to match the audible levels.

 

Example snippets are attached, look in the repository for V1.4.7C.  YOU NEED THE NEW .DLLs in the zipfile!!!   The example snippets are directly attached...   This is the first build on the new environment -- LET ME KNOW IF THERE ARE RUNTIME PROBLEMS!!!

 

https://www.dropbox.com/sh/1srzzih0qoi1k4l/AAAMNIQ47AzBe1TubxJutJADa?dl=0

 

John

 

aldemo-CDorig.flac aldemo-decodedsnippet.flac

Link to comment

About the release -- heads up, and request opinions.  There is a decoding submode option 'b'.  It is different from the old 'b', but has a similar purpose.  When you try to do decodes -- the straight --fcd, --fce, etc etc etc.., if something seems a little odd, try adding a 'b' argument, like:  --fcd=b, or --fce=Gb, or whatever.   I wanna know if the 'b' enabled should be the default.

 

If it helps, use 'b' for now, if it doesn't help -- or hurts, then tell me.  I'll maybe able to remove 'b' all together, and we have simplified things.

 

My hearing is so darned unreliable, especially near the end of the day, I need to ask for help.   I wanna simplify this things, but a lot of the 'options' and 'submodes' come from my inability to decide.

 

JOhn

 

Link to comment

I am not good at double-decoding yet, but just randomly tried the Carpenters example as attached.  (Sorry, just a snippet.)

 

This is PROFOUND.   Apparently, one reason why we have only gotten marginal improvements is that they have been hitting the recordings TWICE!!!   This is very difficult to decode, but some day in the next week, I'll automate it...  Listen to this -- it doesn't even sound like the original at all -- very good!!!   I am NOT expert at this yet, and my hearing might cause me to make some suboptimal choices, but THIS COULD ONLY BE DONE AS TWO PASSES IF THE DECODER IS NEAR PERFECT.

 

After listening to this -- it is proof that we have been getting INFERIOR copies of the recordings!!!!

 

 

04 - Please Mr. Postman-2.mp3

Link to comment

Unhappily, I just read a critical comment about the FA project -- and want to explain what is going on.   Different than lot of projects, stopping and proclaiming success when it becomes difficult...   This project has a real goal and is doing something almost TOTALLY NEW, and never done in the consumer space. 

 

Here is what the project is:

 

1)  The problem has been treated skeptically.  I have caused rifts with people because I persist doing this project.

      Industry does NOT want this project to be succesfully completed.

2)  I have no specification AT ALL to work against, even guessing at the equalization.

3)  The base technology hasn't even been done before (full quality SW DolbyA compatible decoder).

4)  My goal is not 'completion', but the goal is 'perfection'.   It is to leave something after I am gone.

5)  I really NEED help on this.   I can do all of the technology, but my skills are finite, and judgement about

     sound quality is imperfect.

 

I am learning as we progress.  This is NOT copying schematics, making minor changes and then building a circuit.  I have no schematic of the project to start with.  There isn't even a block diagram.  This is a DIFFICULT project, not easy for the smartest engineers/DSP people, let alone me.  Some people have claimed (to me) that it is impossible.   Boy, they were wrong.

 

When people help me, they aren't helping ME, but they are helping people in the future.  This project is a legacy, that might even forward technology in a very small way.   I would think that 50% of analog knowledgeable EEs would know generally how to design a low noise audio preamp, and perhaps 10-20% of them would design a REALLY GOOD low noise preamp.   How many people do you know could do the FA project from scratch, not even possessing a DolbyA unit themselves.  This doesn't make me 'better', but it shows why there are sometimes 'trouble's.   I'd suggest that most people would likely have given up a few years ago.

 

The only way that the project has made progress is 1) my bulldog nature, will never let go.  2) I am not the least technically competent person around. 3) (and just as important), those people to help the project.

 

This is NOT a yet-another-expander, and is NOT a hobby compressor, and does NOT have a fancy GUI -- because the goal is not flashyness and sales, it is something that has NOT been done before.  There is a SPECIFIC performance that must be met -- it is easy to do a 'good' expander...  It isn't so easy to decode DolbyA very accurately.

 

Please be patient with me, and the spits and starts.   This double decoding thing IS a major breakthrough.  Double-decoding isn't ready for prime time yet, but just a few minutes ago, I corrected the 0.11dB error in the output level in the decoder.  The reason is that when the levels are consistent, then multi-passes become easier.   BTW, the original versions of the development code DID support multiple passes, but Richard said that wasn't needed.   Even audio professionals don't know all of the answers.

 

NONE of us knows all of the answers, but I promise you -- this project will be perfect (or as close as possible) when I am done with it.  In fact, the decoding of DolbyA material is already better than the previous perfection!!!!   I think there is a fairly good track record, but frankly FA decoding is much trickier than DA decoding, because there is ZERO basis for correctness other than LISTENING.

 

I was not given a block diagram -- this is TOTALLY FROM SCRATCH new development!!!

 

As is, for single passes, the decoder IS finished...   The documentation is in progress, and so is full multi-pass support is in progress.

 

I like to do truly difficult things, because someone else has already done the easy stuff!!!

 

John

 

Link to comment

Fantastic results and formula for dual decoding -- before the next release, there is a slight mod on --outgain.

The embarassment is that I didn't realize that almost everything is double encoded, including Supertramp, Carpenters, Yes, etc.  Everything.   All of the previous demos and decoding results only provided a small part of the total improvements.

 

------------------------------

For dual decoding, you need to do two passes.  Yes, it really slows things down, but also you don't need the higher quality modes for the profound improvment (the higher quality modes only remove the mellow fog.)

Pass one:

da-avx --input=infile.wav --overwrite --outfile=tmpfile.wav --fcd --tone=-13.50 --fw=tpop --normal --xf=7 --hf1off --outgain=0.11

Pass two:

da-avx --input=tmpfile.wav --overwrite --outfile=outfile.wav --fcd=G --tone=-13.50 --fw=tpop --normal --xf=7 --mfoff --xof=1.19

-----------------------------

This above, dual pass decoding will work with the currently released version of the decoder.  There are some potential variations, maybe some recordings prefer --fce instead of --fcd=G on the second pass.  Also, you might try just --fcd without the =G.   The '--xf=7' tunes up the input detector, but just skipping that switch, or using --xf=5 will speed things up a little, yet still give slightly superior quality.

 

Note the use of 'tmpfile.wav' for the file sent from one run of the decoder to the other.  You can keep the file, but it isn't really worth it.

The --outgain=0.11 will go away in the upcoming release.  Also, the upcoming release has a slight tweak on the --fcf type EQ -- don't worry, probably cannot hear the difference.

 

All of these demos were done with EXACTLY the above settings (well, I am using a slightly newer decoder, where outgain isn't needed.)

These are PROFOUNDLY improved...   Also, as you can tell from the command line, not using the super modes of the decoder.

 

I intend to build-in dual decoding in the next week.  This is ASTOUNDING!!!  It really suprised me, and  I just realized TODAY that EVERYTHING was dual encoded...

 

John

 

casinoroyal.flac carsdemo.flac oliviademo.flac beatlesdemo0.flac carlydemo.flac

Link to comment

@John Dyson

 

Can you explain very simple what this SW is ?

I suppose some sort of DSP ?

 

Only PCM ?

Can it be on the fly decoding?

Will it work with MQA ?

Can it be implemented into Roon or HQPlayer (or Sonore SonicOrbiter Linux) ?

High power demands, or can even work with a Pi ?

Is your SW mature enough to have it reviewed maybe here at AS as a start ?
 

I saw a project where you need the original tapes and you and a Canadian guy where able to get better AD conversion. I guess this is a spin off from that project ?

 

 

Link to comment
9 hours ago, R1200CL said:

@John Dyson

 

Can you explain very simple what this SW is ?

I suppose some sort of DSP ?

 

Only PCM ?

Can it be on the fly decoding?

Will it work with MQA ?

Can it be implemented into Roon or HQPlayer (or Sonore SonicOrbiter Linux) ?

High power demands, or can even work with a Pi ?

Is your SW mature enough to have it reviewed maybe here at AS as a start ?
 

I saw a project where you need the original tapes and you and a Canadian guy where able to get better AD conversion. I guess this is a spin off from that project ?

 

 

This is a long answer to your questions -- I think that this mostly qualitatively answers your questions, but not every point.  THis will give you a sense of what the software is and is about.

 

The software does two things:  can decode old DolbyA tapes (very very very well), and clean up consumer recordings, often damaged by DolbyA related compression.  There is amazing progress being made for cleaning up the consumer recordings, and I just found some major breakthroughs for further improvement.   It is likely that the AUDIBLE improvement using this decoder is MUCH MUCH MUCH greater than trying to gain benefit from MQA 'unfolding' or 'undistorting'.

 

Un-decoded MQA is not an impediment, because this program is dependent on the MASTERING of a recording, not related to the data encoding.   Since MQA basically adds noise and distortion to a pure 16bit transport signal, this project will likely help clean up some of that damage.  The problem is when there is no access to the decoded MQA stream, and no decoder is capable of using this decoder as a plug-in, then this program cannot be used to improve the usually mastering damage of most pop, some jazz and classical consumer recordings.

 

The project discussed here is the 'consumer decoder', but the software is *primarily* a decoder of DolbyA material, and that is the main part associated with the 'Canadian Guy', RIchard.  He is not  associated with the 'FeralA' decoding side of the project.  My original goal (long ago) was to undo the very consistent, almost standardized damage to recordings sold to consumers, and after some research, I found that the 'damage' has the characteristics of compression associated with DolbyA.

 

After many, many poor attempts at 'undoing' the DolbyA damage, my project partner 'Richard' joined helping with some of the more detailed idiosynchracies of DolbyA.   Also, during this time, I realized that doing a 'better' expander than DolbyA is the WRONG WRONG WRONG thing to do, but a precise emulation is critical for accurate and clean results.  So, I implemented a super-accurate emulation of the DolbyA characterstics when used in expansion (decoding) mode.  This is a very non-trivial exercise because the design of a DolbyA while encoding is a parallel set of 4 jFET (or diode) compressors with a bandpass filter, but the decoding method for real DolbyA HW is using those feedback compressors (along with bandpass) into another feedback loop.   Another complication is that two of the compressors share bandpass, so they interact with their natural time delays being a factor in the interaction.

 

So, with that very algorithmically complex set of HW, instead of wasting time doing an impossible audio feedback network in software, I painstakingly calcuated the state of everything in the compressors when they are in expander (decoding) mode.  Instead of implicitly decoding DolbyA by using an encoder in a reverse feedback loop, the DHNRDS does a super accurate emulation.  The emulation was 'imprecise' and not directly copied from the DolbyA design, but was inferred from the general characteristcs and the curves were measured on actual HW.   Of course, the measurements were necessary also because of the tweakiness of the HW design, and the selection of components.   Also, the Sony Patent describing a potential 'DolbyA Decoder' was so far off and inaccurate as to be useless.  It was also a kind of parameteric feedback design, where the DHNRDS design improvements become totally impossible.  The Sony patent descriptions of the attack/release calculations were also totally ludicrious.   The DHNRDS is so accurate that it can do multiple layers of decoding the HW created material, and gain accurate, clean results.

 

On top of the super accurate emulation, there are numerous design enhancements including Hilbert adjuncts to the attack/release calculations (mitigating all IMD from gain control modulation), and also the gain calculation itself uses Hilbert tricks to move the modulation compnents temporally to be less audible (mitigating the 'DolbyA fog').

=====

The FA (consumer miode) software is just barely starting to show its fully capabilites.  The SW doesn't 'crash', but is still 'green', but the DA portion (decoding DolbyA materials), it is ROCK SOLID.   FA is still a moving target.  I *STILL* don't even fully know what FA is, but I am zeroing in.   I just had a major revelation over the last few days, and it is both heart breaking (because I thought that the decoder was fully complete), but also encouraging (because the sound quality can be significantly better yet!!!)

 

=====

(More project background, wrt progression/growth)

As the DA decoder emulation was being developed, I was also looking into cleaning up the consumer recordings with the DA decoder as the basic element.   Actually more difficult than the exceedingly difficult decoder (much cleaner than a true DolbyA), is reverse engineering the correct EQ to undo the lousy compression on consumer recordings.   This 'FeralA' project (feral, meaning it shouldn't have been entered back into nature :-)) is a variant (some people might believe deviant) of the DA decoder project.

 

This FA project has been very slow and painful, because I have not clues and been given NO clues by people in the industry about this nefarious compression added to consumer recordings.  NO-ONE in industry is happy about this project, and I have gotten strong pushback.   I would quit, except I KNOW that I am right to expose this, and enable TRUE AUDIOPHILES to fix the recordings that they have paid money for.

 

======

 

The algorithmic complexity of the DA decoding software is two fold:  1) the pre-planning of the pure DA decoding algorithms (very non-trivial, no intellectual property encumbernaces, no patent encumberances), 2) the anti distortion mechanisms, one is super proprietary and I will not likely release the technology until a paper is written.   The FA decoding has one more complexity:  determining the filters and configuration needed to undo the damage to almost ALL consumer recordings.

 

Once the algorithms and parameters of the DA decoder had been determined, the complexity of a simple decoder now isn't all that great, however it is incredibly detailed, even with careful time synchronization for every little filter, including minimum phase attack/release delays along with the easier to deal with linear phase.  Copying and using the code wouldn't be very difficult, but maintaining the algorithms would not be for the faint-of-heart.

 

======

I am willing to license the simple version of decoding technology to a plug-in writer.   The license would be gratis, but would also have proprietary restrictions.   The technology to do *simple* decoding of old DolbyA recordings can run several times faster than realtime on one core of a core2 or Ryzen processor.  The more complex anti-distortion aspects can barely run realtime on 4 cores with a 44.1k source material.

 

======

The program is written in C++, and uses nothing that is license encumbered (other than acknowlegements.)   The only association with GPL is OS oriented.  The SW is as unencumbered as it can be.    I doubt that the SW can be written efficiently in Java, Perl or other such interpreted languages.   The code takes full advantage of CPU SIMD capabilities, and re-implementing in C would be very very painful. When the program is running, 95%+ of the time, it is using SIMD instructions.

 

=====

 

The software is one-of-a-kind, and is the only way to get pristine copies of what was recorded onto DolbyA NR tape, but also can correct (progressively better and better) the poor qualiity being sold to consumers.

 

John

 

Link to comment

Announcement about the 2 pass revelation.   I have observed that most of the time, the encoding process IS done twice.   This means that a single pass of the decoder is suboptimal, but sometimes might be useful.  Given that fact, I intend to spend the next several days to extend the decoder to do 2 passes directly.  The code is modular, but the hard part is determining how to splt the decoding process.

 

I will not be doing an interim upgrade, because it will just waste your time and mine.  SImply add the --outgain=0.11 to fix the level issue, and that is all that is needed to get the astonishing results from two passes.

 

For example, should both passes support the anti-MD, or should the anti-MD only be used on the first pass?  There is a real reason for this possibility, other than extreme CPU usage.  I will be spending 1 day to think, and figuring out the correct combination.  It will require 2 days to make the updates, and a day to experiment with the final choices.  Sorry about this, but it will be Friday before this code is complete!!!  The example a few posts ago will work, even after the upgrade.   I suspect that there will be a 'fixed' capability where most of the two pass work is done for the user.  This has to be done SMART, not just hack a solution.

 

John

 

Link to comment

Clarifying a misconception -- not all recordings use the FA compression scheme in multiple passes.   Some are single pass, but a frustrating number of recordings (Al Stewart, apparently the Beatles remasters, The Cars) all appear to be multi-pass.  The early Carpenters stuff doesn't appear to be multiple pass.  Even for material that was compressed 'twice', a single pass with the currently released decoder is a significant improvement.

 

I have done the low level changes to the decoder for modularity reasons, and also created a  *2pass* mode that supports almost full anti-MD decoding quality in realtime on a 4 Core Haswell CPU.    The difference between the slightly lesser anti-MD quality and full anti-MD quality is audibly NIL.  The difference in CPU usage is 2X faster.  (The modularity in the decoder makes it easy to cobble together different combinations of the building blocks, including multiple sequential decoding operations.)

 

The new 2 pass mode will be simplified, and a very simplified 1 pass capability wil also be included.  This will include the faster 'full quality' decoding capability.  The new, faster 'full quality' mode is good enough that I'll probably regularly use it.  The 'ultimate' quality only makes a very slight improvement over this new mode.

 

John

 

Link to comment
23 hours ago, John Dyson said:

the software is *primarily* a decoder of DolbyA materia


Does your  SW decoder understand that what’s  Dolby A or not ?

I’m asking cause my understanding is that Dolby A was only in use a certain period from 1965 to I don’t know.
 

Do you think it can be implemented into Roon ? (Or other players).

(Roon will not accept VST plugin). 
 

What process power is required to decode on the fly ? ( I think you already gave an answer: 4 core Haswell). 

 

You use the term consumer recordings. Is that only red book or anything available on Tidal an Qobuz ? Or did you say that MQA (as in Tidal) actually make your SW useless ?

 

Do you see this SW something DAC manufacturers can use at some point ?

 

 

@Miska

I must say I don’t totally understand everything in John D answer, but I think you do. 
Do you think this decoding SW is something you can implement in the future of HQPlayer. 
(I hope you at some time talk to John and discuss possibilities). 
 

 

Link to comment
5 hours ago, R1200CL said:


Does your  SW decoder understand that what’s  Dolby A or not ?

I’m asking cause my understanding is that Dolby A was only in use a certain period from 1965 to I don’t know.
 

Do you think it can be implemented into Roon ? (Or other players).

(Roon will not accept VST plugin). 
 

What process power is required to decode on the fly ? ( I think you already gave an answer: 4 core Haswell). 

 

You use the term consumer recordings. Is that only red book or anything available on Tidal an Qobuz ? Or did you say that MQA (as in Tidal) actually make your SW useless ?

 

Do you see this SW something DAC manufacturers can use at some point ?

 

 

@Miska

I must say I don’t totally understand everything in John D answer, but I think you do. 
Do you think this decoding SW is something you can implement in the future of HQPlayer. 
(I hope you at some time talk to John and discuss possibilities). 
 

 

There are two modes for the decoder, a true, accurate, dead-nuts on decoder that decodes DolbyA.   It actually avoids a lot of the 'DolbyA' fog (well known by old timers) and is better at recovering the info on tape.   Basically, the on-tape info is really clean, but the old DolbyA HW didn't decode very accurately.   The super high quality modes are very CPU expensive, but the moderately-improved decoding is something that I could provide reasonable complexity source code for someone to re-implement into a plug-in.   This would likely be the most complex plug-in that the writer had ever written, even in the moderate (just better than DolbyA) decoding mode.   For an over-simplified idea of the needed complexity, refer to US patent 5,907,623 -- but that is, in reality about 1/3 of the needed complexity (the attack/release calculations are totally missing).   The DHNRDS does not infringe at all, but it gives the idea about what is going on.  (Again, the attack/release calculations are about 2X as complicated as the simple feedback design example in the patent, but I used an unfolded feed-forward design -- much more opportunity for improvements.)   The source is C++, written in SIMD idioms.

 

Next, there is a ubiquitious use of DolbyA encoding for recordings distributed to *consumers*.   I call that compression morass on almost every consumer digital recording 'FeralA'.   Decoding this material entails proper EQ and then applying a DolbyA decoder.  Perhaps 1/3 of the recordings require 2 passes to bring the recording down to baseline.   It is this 'consumer' decoding that is still being improved with my friends on the AS forums.   It is close to finished, but still needs some testing.

 

I will be willing to license (for free) the algorithms for the 'consumer' FA mode decoding also.

 

The only thing being held proprietary is are the very high quality anti-MD algorithms.  There is NO prior art and this is a totally new algorithm.   I intend to release this technology when the time comes.

 

SO, the moderate (better than DolbyA) decoding technology would be licensed gratis to plug-in developers, but don't underestimate the work -- the DHNRDS is a true precision emulation, and not a 'filter scheme ' like ALL OTHER DolbyA plugins.

 

The FA scheme is coming very soon, but the basic implementation exists now, the hard part for me has been to make the UI (user interface) practical, and to find the exact EQ being used.   It has been a long walk.

 

My goal is NOT money, but the goal is that the information/technology be available.

 

John

 

Link to comment

There are two upcoming releases of the decoder, and a mention about the multi-pass decoding.

 

Soon (within a day or so), there will be a new decoder release that is much more precise, in fact, it will also be a DA release on the commercial side.  The big difference is in accuracy -- even 0.25dB error can quickly become in the order of 1.0dB error with multi-pass, and also the attack/release was made even more precise.

 

In addition to some basic quality improvements and a few major basic efficiency/accuracy boosts, the high quality decoding mode '--xtra' (the lowest of the high quality modes) is now razor's close to the highest quality mode '--xpp'.

 

This new basic quality improvement will NOT need further documentation, even though the previous documentation needs upgrading (I am too busy while working on the software, and couldn't even merge @lucretius work yet.)  There are no command line changes to get the quality improvement.

 

The forcing function for the quality improvement comes from the two pass deocding, and my notice of some sloppiness in the 2pass  results.  WIth the new basic changes with a release hopefully tomorrow/Sunday, there will be a two pass (all built-in) release on approx Tue/Wed.

 

I am not strongly encouraging that everyone necessarily try to do two passes, even though single passes are much simpler than they used to be, two pass is still fraught with error.   Just now, I am starting to get really good two pass results -- I have some Olivia Newton John stuff that will literally blow ANY fan away.  On the other hand, there is a silk purse from a sows ear syndrome with many other recordigns, where the effort to do two passes isn't worth it - the significant technical improvement beyond 1 pass isn't really beneficial to the listening.

 

My London Philharmonic reocrindings have turned from 'woody and compressed' -- to stunning after two passes.  So, some recordings can become stunnning, but others -- not so much.  If you get a good one pass decode, that alone can stil be a great improvement.  Two passes on the correct recording can blow-you-away...

 

I am working at least 12Hrs/day on this, even when being a little under the weather -- this is seriously good stuff, but I have some reservations about increasing expectations for two passes.   The new one-pass results ARE improved, but the improvements were really needed to get repeatable, best quality two pass results.  When the two pass results do show major improvement, that improvement can be worthwhile, but don't spend an hour on a single album...  I mean, if it only takes 5 minutes to choose the best parameters for one pass, but 20minutes to line everything up for two passes -- it MIGHT not be worthwhile to spend the time on two passes for every recording.

 

It will be a few days -- and this new version that I am testing right now, there is almost no error that I can measure.   There have always been 'little', 'small' differences, and there are still some minor frequency response oddities, but the new results are far better.  Decoding FA cleanly is MUCH more difficult than doing DA material.  The robustness is greatly improved.  The dynamics (the really hard stuff) are very much improved.

 

Working very hard on it.

 

John

 

Link to comment

Before the release tonight or tomorrow (hopefully earlier rather than later), I wanted to describe the difference in behavior and why an upgrade can be helpful...   You know about all of the anti-distortion mechnanisms, but one thing that I had always done, is to bias towards a more controlled attack/release (esp attack.)   Sometimes, in my bias, I slowed down the attack a little too much, attempting to avoid the spread of distortion (intermodulation) sidebands.

 

A lot of the code has been reworked in the last several days, including a review of the responsiveness of the attack/release.  Previously, the attack also had to be slowed down because of the somewhat 'clunky'  HF0/HF1 cancellation.  I found a more efficient/more nimble way of implementing the HF0/HF1 cancellation.  Previously, it was approx as slow as the original HW design, but the new version has shorter  propagation delays, and as such allowed me to loosen up some of the attack dynamics, therefore giving an even more clean/clear sound.  Part of the work-around for distortion is needed because of the HF0/HF1 cancellation, but that is now effectively 100% effective, and easily controllable.

 

When the new improvement in the perplexing HF0/HF1 cancellation -- a major sound quality problem generally in all DA implementations -- and the ongoing improvements in anti-IMD and anti-MD, there is an significant improvement in clarity.

All of this, of course, is actually in preparation for the full quality multi-pass support, but the improvements have a side benefit for normal single-pass decoding.

I still might be on schedule for the upgraded single-pass release tonight, but I am being very careful, especially this time.   It might be delayed again till Sunday, but I am really looking for imperfections, and still chasing down very minor problems.  I haven't heard any new nits in the last day or so, since we have an opportunity for major improvements, the sound quality is being CAREFULLY scrubbed.  One of my objective tests is still showing an anomaly, not major though.  If I find an opportunity to correct the problem (a slight, short term overshoot in the attack in very limited situations), I will fix it before the release, but does require careful testing.  (The overshoot is not actually in the attack, but is caused by an overshoot in the encoding DA, that the DHNRDS does not completely compensate.)

 

This is the first opportunity for a significant DA decoding re-engineer in a year or so.  (No, I do not re-imagine anything, that ends up being a random walk into oblivion).  This is a real safe and conservative rework, looking for any way to improve the behavior, and listening for any nits that are caused by 'decoding'.

 

John

 

Link to comment

It has taken a lot longer to upgrade the audio on the upcoming release.  There is still a file permissions problem.  I have provided some examples to the PM group for review...

 

1)  the anti-MD and anti-IMD had a bug in the time delays, where the lows were incorrectly delayed longer than the highs.  This was more of a minimum-phase vs linear-phase bug, where the processing should NOT have been based on frequency, but instead should have been more constant delay.  This has allowed complete unleashing of the goodness of the anti-MD and anti-IMD.

 

2)  some of the emulation of the DolbyA has been dropped, specifically the feedback loop delay.  After reviewing the frequency response curve of noise vs. spectrum of the true DolbyA, and thinking carefully, I believe that some of the excessive peaks on true DolbyA decodes come from the feedback loop delay.  When I removed the delay on the DHNRDS, the peaks flattened on it also -- to me, the sound is much more transparent, and cascading two decode passes is now much more stable.

 

3) the code has been verified at TWO decoding passes on material that demands it.  Previous versions were sometimes a little unstable on certain transients, but the instability is now remedied.

 

Most of the improvement is more noticeable on multiple-passes, but the single pass behavior now is a greater improvement from the old decoding than the decoding used to be by itself.  That is, the improvement in clarity on difficult material is more revealing than decoding by itself previously was on the old version.

 

It will take two more days to remedy the permissions problems.  I think that I know what it is -- a combination of zip maintaining IDs and the emulator using those IDs.   I need to remove any permissions from my base files so that they don't get transmitted through the distribution.   Windows is weird -- it does too much behind the scenes.

 

I cannot publically distribute the full examples, but will resolve some examples to snippets in a few hours.  I really have been busy, and the improvements are creating results that never sounded like DolbyA encoding ever touched them at all.

 

John

 

Link to comment

I just had a crazy wild idea for some of the internal gain filters on the DA & FA decoding -- why not actually use linear phase FIR filters instead of my IIR contraption (part of the front end anti-IMD code.)   I stripped out all of the IIR filters, and added some short FIR filters -- WOW -- much more delicate sounding.

 

The previous demos that I mentioned -- the results are even better!!!

 

The good news also, with the new front-end, there is less need for anti-MD code, so quick decodes produce even better results.

Still finalizing the new release.

 

John

 

Link to comment

The new FA decoder (and super duper improvement to DA also), V1.4.7S is ready (Linux & Windows.)  There is still a question about file permissions, but I put a hack in that should avoid the problem (I hope.)  It isn't a bug in the decoder per-se, but something odd about the emulation library.

 

There is ALMOST never any need to tweak the decoding parameters very much now, since the two pass decoding actually solves the problems, using totally standard parameters.

 

The decoder works just the same as before, or even better.   Let me explain -- below are three decoding commands, most likely one of a, b, or c will work, and work SUPER WELL.   I am only showing 1 pass decode information immediately below.  There will be a new release in about 1wk with a revamped command line.   However, I'l show a two pass decode as an example ONLY.

 

1 pass decodes: (choose these in order, most of the time --fcf will work VERY WELL for 1 pass.)

a) da-avx --input=infile.wav --overwrite --output=outfile.wav --fcf --info=2

b) da-avx --input=infile.wav --overwrite --output=outfile.wav --fcd --info=2

c) da-avx --input=infile.wav --overwrite --output=outfile.wav --fce --info=2

(choose the correct decoder for your machine:  da-avx, da-core2, da-win)

You can still use --tone= (e.g. --tone=-13.50 is the default), but usually you won't have to do it.

 

2 pass decode (this is knarly for now):  ONLY FOR EXPERTS UNTIL I SIMPLIFY THE COMMANDS...

da-avx --input=infile.wav --fcf --fw=none --wia=2.0 --mfoff | da-avx --overwrite --output=outfile.wav --fcd --fw=none --woa=0.50 --wof=1.19 info=2

 

For each decode (in the single pass or each in the 2 pass), you can add the --fx switch for higher quality (MUCH higher quality.)  You can still use the --xp or --xpp switches also, but they are TOO GOOD.   --fx or --fz are good enough, really!!! If you want better quality than --xpp could ever do in the past, then use --fz=max..  --xpp=max also works, but will take FOREVER.

 

Some recordings might benefit from tweaking the calibration a little, and SOME actually need -12.95, but few really do.

 

There are more super quality options available, but I don't want to confuse matters since the manuals are all out of date.  THIS WAS A HAIR RAISING SET OF IMPROVEMENTS.

 

Expect the most amazing detail that you have ever heard on consumer pop recordings.   Classical that was FA is amazing now.

 

Look for V1.4.7S, in the same repository as before:

https://www.dropbox.com/sh/1srzzih0qoi1k4l/AAAMNIQ47AzBe1TubxJutJADa?dl=0

 

DOCUMENTATION and new built-in simple commands for two pass coming in a few days.  I deserve a rest.

(My previous private demos are very poor examples of what the decoder can do now!!!!)

 

John

 

 

Link to comment

New release:  V1.4.7Y, the last of the V1.4.X series.

 

This is perfect...  full stop.

Previous instructions above....

 

If you want a 'perfect' single pass decode, just use the '--fx' quality switch.  Multi-pass (not yet internally supported) requires more finesse.

 

Just remember, either use the default quality or the '--fx' switch.

This cannot get any better...  This is exactly what I used for some super impressive demos earlier.  I can produce snippets for you, but YOU can do PERFECT corrections of your CD anytime now.  The usage is quite simple now..

 

Here is the location (as usual), and you want the V1.4.7Y version, NONE OTHER.

Better docs and two pass version will be forthcoming soon.   Just ask me publically or privately if you want to try and need help.

 

https://www.dropbox.com/sh/1srzzih0qoi1k4l/AAAMNIQ47AzBe1TubxJutJADa?dl=0

 

John

 

Link to comment

For now, I need to back off on the decoder.  This isn't about pulling away from the project, but there are some things to revisit.  I'll be back after the middle of the week.  I wanted to have this ready for the weekend, but ran into problems.

 

This message is for everyone, including the PM group.  I have had a health problem recently (actually most of my life), and it has gotten out of control.  One of the symptoms is that I am making strange mistakes because of a problem with short term memory (it gltiches, like a seizure.)  This is NOT all about me, but I owe those interested an explanation.  I WILL be okay, and am safe and okay.  I do not want the project to suffer though.

 

I need to sit back for about 1wk, while some meds that I had to start do settle in.  I am playing whack a mole with the most simple things, and what just happened a few minutes ago has hit the breaking point.

 

Also, there *really* needs to be some consideration about freeing the source code now.   At least, the development needs to be more diffuse.   It needs to work well enough (again) to develop more interest for someone else to join the coding effort.  If I cannot improve my reliability, or improve the reliability of the releases, the project future is at risk.

 

I will NOT be gone, but will be working on the project in a different mode.   I am dropping out of product mode for a few -- maybe a week, and taking it easy.  I will be in 'piddle' mode, and hopefully able to correct some mistakes along with do some internal documentation.  I am waiting for the results of some treatment to settle, and perhaps PRAY that the gllitches are mitigated.  (This impacts a lot of things, including my reading comprehension.)

 

I feel so bad about this.

John

 

Link to comment

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now



×
×
  • Create New...