John Dyson Posted January 2 Author Share Posted January 2 After due consideration, I have decided to remove the ABBA examples from the demos. I am a bit of an ABBA fan, but there are real, actual technical reasons for doing so... There tends to be several (at least two) common versions of ABBA mastering, not counting the horrible 'TCSR' type stuff, compressed to 'h*ll'. This decision also comes from my own bias of using the 'best available' or 'most original available' recordings as the source. On some ABBA albums, the 'best available' is a confusing choice. Albums that sound 'most' like normal ABBA are NOT the most original. The 'best' then becomes confusing, especially when the 'most original' will be disappointing to those who expect the 'normal' Polar release. The 'most original' is much less dynamic range compressed than the 'normal' versions of the ABBA releases, therefore sound just a little more 'muddy', but really not. It is just that the 'normal' ABBA stuff tends to be more bright sounding, a bit more bright and 'processed'. I have found that the A/B comparisons have also been very confusing to me, because I also personally prefer the 'compressed' normal ABBA releases. This comes in conflict with my orthodoxy of using the most original. Until I can resolve these matters for myself, feeling comfortable with the decision, I have to remove the ABBA demos. I do plan on producing some ABBA demos in the future, along with explanation about the differences. Basically, I just *temporarily* gave up on the ABBA stuff for now. Why make the decision now and 'give up'? Good stuff is happening, and I don't want the ABBA recordings to hold up any sort of possible release. Again, I am an ABBA fan... But I must do this for the project. John PS: 1) I might reconsider if I can 'organize' my thoughts on this matter. 2) Will probably do a separate demo when things have settled out. 3) Still using ABBA recordings for reference during development -- just have to be careful. Link to comment
Currawong Posted January 3 Share Posted January 3 The Getz and Gilberto tracks had the most stunning improvement of the ones I tried. I'm sure some music is just going to be too messed up to improve. I have a similar problem trying to decompress video. Sometimes it is just too compressed to be recovered. Link to comment
John Dyson Posted January 4 Author Share Posted January 4 3 hours ago, Currawong said: The Getz and Gilberto tracks had the most stunning improvement of the ones I tried. I'm sure some music is just going to be too messed up to improve. I have a similar problem trying to decompress video. Sometimes it is just too compressed to be recovered. Thanks, I agree... A little diversion and then more of an answer... I did just have a startling experience that I mentioned to someone in PM... It is very interesting that I can sometimes NOT hear an improvement upon waking up, then lo and behold an hour or two later, it becomes obvious... The FA scheme is insidious, and once I do the best possible controlled test, the difference is subtle, sometimes a lot, but on some, nonexistant. Most of the time, it is the descrambler that does the work and is most subtle. Much of the time, what I hear is more 'body' in vocals or a more true mix of midrange/bass with HF. SOME RECORDINGS ARE NOT IMPROVED, and all should be enjoyed as-is!!! About some recordings not having ANY improvement, then others some (but mostly not overwhelming)... The two phases are the DA layers, which do NR mostly at low levels. There is a slight improvement in dynamics at low levels, and is a combination of DA and descrambler. At high levels, the descrambler alone is active. The descrambler effect is to eliminate the 'pucker' effect in the sound. The 'pucker' effect can make vocals sound like there is 'hash' around them to those who haven't become FA accomodated. The hard part about the descrambler work (still trying to make it correct) is to make sure that the time delays match, and to get the ratio of LF/MF vs HF. The biggest challenge is my own hearing being consistent ( varying blood flow and probably some expectation bias.) I have ALL of the SW/HW tools necessary to come to a complete solution. The decoder does NOT resolve all issues with recordings, and might not resolve all issues about FA, but seems to give some improvement (depending fully on how the recording was handled.) It is this SOMETIMES and my own expectation bias that keeps me from doing a release right now. I feel like the limitations need to be resolved the best possible. Understanding the details better will help me feel more comfortable about my own understanding AND integrity. ( The delay this week has been personal, and I will be more active again well before a week from now.) There will be a release soon, but only after I can find the proper timing (yes, the descrambler can muck up the timing between LF/MF/HF) and deal with my expectation bias more effectively. FA accomodation has also been a challenge, in the last few years it has required week long vacations to be able to hear the worst of FA again. There IS significant action in the descrambler and improvement, but I must find the best way to deal with proper A/B in a more practical way. I intend a release in the short-weeks timeframe, even though I had suggested it might be earlier. Accuracy, and having personal integrity is paramount. I'll do an upload very soon. (A medical condition has my hearing all over the place, I expect it to be stabilized soon. Doing the precise A/B requires very stable hearing and a good 7-10 second detailed memory. ) Answers are coming, hopefully useful improvements also (the decoder is way too slow, and decreases it's practicality.) Thanks again for your comments!!! John Currawong 1 Link to comment
John Dyson Posted January 12 Author Share Posted January 12 Quick status: Progress might seem slower because of recent quiet, not much messaging very much. A LOT of zeroing in work is being done. Actually, the decoder is structurally complete, but the descrambler is currently doing 'too much'. There has been no need to change the structure, however some 'tweaks' have been done as glitches in behavior are found (e.g. minor problems with rate conversion.) The descrambler is a lot more complex and a lot more tedious settings than ever estimated. Once the dynamics finally seem correct, I'll be asking for opinons. ANY TIME, if anyone is curious about 'proof of progress', just tell me and I'll find a version that can be demoed within a day or so. Full demos require a lot of time, and when a demo sequence is run and a bug is found, then the problem is resolved. Adding the demo phase can significantly slow down the development -- which is going VERY fast -- I mean, VERY VERY fast towards accuacy. The development is much more productive than in the last recent few years. Adding a demo upon request instead of automatically doing the public demos can save a lot of time. -------------------------------- Demos/tests being run: I just did a run of Fleetwood Mac Rumors and the 1970, 1971, 1972, 1973, 1975 and 1976 Carpenters albums, along with some random checks of the 1977 and 1981 albums. Karen's vocals are good and strong, but a bit too strong. The sound is currently similar to an uncompressed studio mix, just too strong for normal listening. Previous results that were too dynamic did NOT sound like a studio mix, and pushed the midrange up too strongly. This is the first serious attempt in the last week or so, but has just been scuttled tonight. The correction is most likely a change in a certain parameter. This 'overly strong' dynamics happens when doing A/B comparisons, finding that the dynamics need to be increased, but there is 'creep upwards' where the sound pattern being optimized becomes too strong. It seems like a version of audio perception accomodation. Apparently, I am very susceptable to it. There will probably be another try starting tonight. The decoding sequence is currently: Fleetwood Mac, Rumors. The Carpenters 8 album sequence above. Certain Linda Ronstadt albums, both from the normal pop and 'big band' recordings. 8 ABBA Albums (very carefully chosen source material) The often used 95 demos.... Aja/aja, Various classical recordings, and more... A lot of material is used for review. The test list is sometimes 'shuffled' and test recordings are also randomly selected. So, right now, the result is incredibly interesting, but not good enough. HF dynamics are just a little too strong. John Link to comment
John Dyson Posted January 13 Author Share Posted January 13 Good news is that there HAS been progress, but the new or deeper bugs seem to appear just as a set of demos has been created. There is still yet another a 'layer of onion' that needs to be traversed. Two bugs known right now: * There is a 'birdie' effect, either coming from timing delay shifts or Nyquist wrap around. The birdie isn't extremely strong, probably not noticeable without the very significant improvement in detail. * The anti-fog is interacting with a currently unknown 'something', perhaps a minor error in the Hilbert calculation. If the bug is what I think that it might be, it should have been correct for a very long time. I am still investigating. The demos havent gone beyond the reviewers, and not worth your time at this point in the development. As of 8 Hrs ago, I didn't know about these bugs... Well, there was a small 'birdie' effect, but is actually worse than I had hoped. You'll be getting something that I feel to be perfect (or a slight epsilon from perfect.) From someone very frustrated, just not today -- but SOON. John Link to comment
John Dyson Posted January 20 Author Share Posted January 20 Very short, but important status report: In most ways, the descrambler is finished, working and appears to do exactly what it is supposed to do. The 'release' failed a few weeks ago, something STILL not working crrectly. The delay in the last few days has resulted from a new layer of functionality, just added to the working developmental code... After a lot of tedious investigation, the 'missing' processing step is 'phase randomization' correction. After an exhaustive set of tests, applying what was learned in the past about the suspected phase swapping, a solution to the time shifting/phase swaping is now being tested. Apparently, the phase randomization is implemented at 110/440Hz, 440Hz/1.5kHz, 1.5kHz/2.5kHz and 4.5kHz/7.5kHz. This would have the tendancy to decrease the strength of peaks on average. Details have been iteratively determined, and still being double checked. As soon as I am happy and have gotten helpful re-enforcement for the improvements, I'll make the demos available. The updated experimental decoder is infinitely better than anything publically (or privately) demoed. I won't be 'holding out' much longer!!! John Link to comment
John Dyson Posted January 22 Author Share Posted January 22 A new demo version has started being made available. This is pretty much finished and about as good as it can be (plus or minus some tilt issues -- slight HF/LF boost/cut.) I found that the correct descrambler settings happen along with the elimination of 'birdies' and other kinds of distortion. This is NOT the release yet, but is getting very close. This MIGHT be 1-2 lines of code away from 'release', simply to make the final adjustment to the HF or LF tilt. There might still be a bit too much HF, or a slight tilt error, might even be <1dB, but subjective hints will really help. I have attached a frequency response table that shows that the STATIC response is essentially flat (acknowledging >+-40+dB gain control), but of course, insufficient to describe the actual dynamic response!!! * Any straightforward hints that target a specific problem will enable a <1day turnaround, with tight controls on the change. The last month or so has been wrapping up loose ends and desperately trying to help control the variability of my hearing for during the A/B comparison iterations. https://www.dropbox.com/scl/fo/mkku7cqtsw960nxmt6by6/h?rlkey=t378odgg5cudiuouve5jkgh7c&dl=0 Try the normal demos first. The ABBA snippets and Carpenters snippets might not produce quite as unbiased results as the 'normal demos' as described below. The normal demos include some classical/instrumental materials, while the ABBA/Carpenters are 1970's pop vocals with the associated sensibilities. Even the more complete 'demos' (flacsnippets/fscmp10/fscmp35) aren't enough to give a fully accurate impression, but is as good as I can provide from my own archives. The demos contain 10 second A/B/A/B and 35 second A/B/A/B comparisons. (The first is 'decoded'.) fscmp10 -- 10 second segments. fscmp35 -- 35 second segments. flacsnippet -- single snippets alone. I believe that there might be a bit too much presence in the sound, but the behavior of the descrambling is very very good (that is, NO distortion.) Also, it seems like the 'birdies' are gone, which means that the distortion creating the 'birdies' is now gone. The Carpenters & ABBA snippets were included because of my own personal interest and ongoing discussion with a special ABBA fan. If I could control my perception and get repeatable results, the 'final' version would be available right now. The flaws are NOT technical because the decoder/descrambler is *TECHNICALLY COMPLETE AND FULLY FUNCTIONAL*, but there are some excruciatingly fine settings that I am having troubles determining because of my very unreliable ability to judge. HINTS at this point are very helpful. Those not in the specific discussions might be surprised about the circumspect feedback and my ability to interpret and make corrections based on well intentioned comments. If you can hear 'what you like' with a bit of a 'tone control', perhaps a bit of HF or LF shelf on the audio-range endpoints, then it would be very helpful to let me know your suggestions!!! as always: THANKS!!! The decoder is essentially *TECHNICALLY COMPLETE*, now needing minor polishing, major improvement in usability and I NEED TO PRODUCE BINARIES FOR WINDOWS USERS. John resp-V20E-RELTRY9.txt Link to comment
John Dyson Posted January 23 Author Share Posted January 23 The feedback has been good (no change) so far. I apologize to the PM contributors that I hadn't contacted, but this release is really worth it. To everyone who has helped in the review and development thanks, because the listening to the results of appropriate material fully justifies the concept (but not the effort, because it was TOO MUCH WORK.) The FA encoding/distorting scheme is essentially proven even more strongly than before. The old 1980's 'digital sound' is still with us, but now effectively removed by the FA decoder!!! Here is the Linux version of the decoder (source available anytime on request)... Windows version coming in a few days: https://www.dropbox.com/scl/fi/022ppa6718elxs1pgx94e/da-V20E-RELTRY9-Linux.txz?rlkey=2ns04adnxkbobfm6ecwkx33l6&dl=0 PS: the decoder output IS an improvement, whether or not it is worth the trouble? I cannot answer that question. The challenge over the last month or so has resulted from my stubbornness about not needing extreme accuracy/precision. There is NO leeway for error, even though from a normal 'gain control' background, it might seem like small errors are okay... That is, slightly wrong choices in the settings, perhaps to give a 'better sound' will only produce 'worse sound'. When 'wiring together' the descrambler steps, the analogy is more like 'binding together line segments with hard nonlinear transistions' instead of a more 'elastic' thing like a normal mult-band gain control might act like. Until I fully understood that every loose end has to be tied up with the descrambler and every setting must be correct, or there will be a 'creeping crud' in the sound. There are more than 20 bands, perhaps as many as 40 or so if you count the 'pilot frequencies'. Each of the bands must be set correctly, but luckily the settings are 'discrete values' instead of 'tweaked'. Sometimes, errors produced 'birdies', sometimes a 'grainy sound', sometimes both. It seems like the continuous set of iterations, finding the optimum case where there are no birdies, no distortion, etc has been successful. I did learn about a few more 'discrete values' needed to produce accurate results instead of the previously guessed at values. The 'sqrt(2)' Q value is interesting, because it can be each of '1.414..., 1.42 or 1.37... ' What is this 1.37xxx value? It is actually '2.213^0.4', and also ends up being related to a combination of a Chebyshev & Butterworth filter together (Q=0.8409/Q=0.707). It is interesting that all of the 'magic numbers' end up working smoothly together. Of course, at this level it might seem like 'numerology', and in a way it feels like it, but really all of the numbers are interconnected, and VERY important to use the correct Q values. Until a week or so ago, I didn't know about the 2.213^0.4 Q value, and had been very frustrated wondering why the equalizers (shelving filters) weren't working quite correctly. Once all of the correct values had been found, the sound became more and more profoundly clear. There appears to be nothing else that needs to be determined. The descrambler SEEMS to be correct. A big relief -- the descrambler concept that uses ONLY shelving filters for processing elements, can do a good job of gain control... Time to write a paper!!! I don't expect any fundamental changes. No matter what, I am always listening/hunting for possible improvements. No real changes are expected until the Windows based FA decoder is available again. Starting tomorrow or day after, I'll start on the Windows port of the decoder. It had already been 'ported' though, just that the development environment needs to be re-created. In the meantime, I am sitting back and listening to the decoder output. John Link to comment
John Dyson Posted January 23 Author Share Posted January 23 This is a response to a recent comment from an active reviewer -- this 'maybe' critcism will not cause a delay, and if there is a change it will be very very minor. (The change already exists in the developers 'quiver'.) I have been getting feedback that the quality is 'good enough' to wrap it up, but the the potential change is so minor that it would be regrettable if the correction is needed and we don't make the change. The errors are becoming VERY SLIGHT, if existent at all!!! My response to an potential, not 100% determined observation about slightly too much 'presence': I agree with you about the possibility of too much 'presence'. It had been worrying me, but also not trusting my own perception. I have a correction for the presence (per my subjective perception at this time), without affecting the actual freq response. The change is 'one step' towards more lowerMF/upperLF dynamic processing (essentially a kind of 'bass boost'). If we get a consensus and if your intuition remains the same about maybe 'too much', the very smooth and clean correction is ready to go. We are really only talking about 1/2dB to 1dB transient response difference, but it can happen at just the right time to be audible. The static freq response shouldn't even show a 0.01dB difference (the difference in sound vs. the static response not changing is amazing to me!!!) The target is approx two days for the Windows version being available, but being realistic it might be a day or so longer than that. The 'port' already exists, just that I cannot build it without the development environment being re-created. Thanks to everyone, and I PROMISE THAT ANY maybe-CHANGES ARE MINIMAL AND SO FAR ARE READY-TO-GO!!! Honestly, other than the very minor, possibly slight excess presense, during testing, I otherwise cannot hear ANY bugs and have been listening for anything that I can find. Completion is far beyond hopeful, simply that technical completion might be one or two days longer. I can forsee NO technical changes that are likely at all to cause a week delay. If the situation is as I predict, the change only needs to be finally verified and the demos recreated while I am working on the Windows development environment. John Link to comment
John Dyson Posted January 23 Author Share Posted January 23 The quandry about the sound having a little too much presence has been decided. Apparently, in V20E-RELTRY9, the descrambler activity in the upper bass is too weak. Also, along with being too weak, the timing is dynamically shifting a little -- thereby allowing a little excess garble into the sound. That is, the garble in the FA version, not normally all that audible, is allowed to become more obvious in V20E-RELTRY9. Even in V20E-RELTRY9, there is no increase in actual garble, just that the clarification makes other flaws more audible. Doing a direct A/B, by my perception, the expected V20E-RELEASE0 decoder sounds just a little more warm and during vocal chorus, the highs are a little more pure. There is definitely some mitigation of excess presence, while improving the already good clarity. In preparation for completing the initial command line Windows port, a full set/rerun of 'demos decodes' are strarting in an hour or so. RELEASE0 will be used to create the same set of demos as the previous RELTRY9 demos. I had hoped that 'RELTRY9' would actually be the same as the release, but perhaps I was being mildly over optimistic. After running for a few hours, I'll be checking the status of the demos runs. There'll be a few spot comparisons with the RELTRY9, original FA and a few older decoders. As long as there are no serious flaws, then the RELEASE0 decoder will be demonstratively the IT release. If there is a need for a modification when completing the port to windows, the release ID will need to increment to RELEASE0A, otherwise the decoder will be frozen. In case I had made a mistake, there might be a RELEASE1 instead, but not planned or expected. The decision point for needing a RELEASE1 would come in just a few hours from now. This plan is the result of PM and email discussions, being very very sensitive about needing to bringing the development to completion!!! I want to be able to declare, without reservations, 'C'est finis', more strongly than anyone else in the world!!! VERY LIKELY, the announcement will be made later this week/early next week. The decoder *REALLY DOES* sound pretty good... John Link to comment
John Dyson Posted January 25 Author Share Posted January 25 The cleaned-up (final) version should be available after the weekend. Got some very useful feedback, nothing fatal -- just minor/helpful. During the code-review during making the slight 'presence' modification, I found a pre-existing overt bug in one of the high frequency discriminators. There is one heck of a lot of code in the descrambler, lots of it looks similar. There was a simple error in arithmetic sign in one of the discriminators, which can/does cause some distortion. The situation wasn't major, but just wanted to be very clear about necessary changes being bugfixes. I should have made the descrambler even more table-driven, but it was designed/redesigned/etc, and left some cruft in the implementation. The 'presence' change was a little less 'minor' than the 'discriminator bugfix', but on the order of a few lines of code in one specific area. The actual change was to increase the processing in the lower midrange, thereby making the presence area sound less pronounced. The result is a more 'flat' sound. Like usual, reading the frequency response shows not much of a change, but the subjective difference is significant. A/B matching is done between the FA and decoded, along with two examples of known never-FA recordings used as reference. As always, I am definitely looking at improving CPU usage, and have had SOME success. Along with the small improvement in CPU usage (fewer cores needed, but still takes significant realtime), the required math precision is slightly less demanding. I'll check to see if some transforms math can be changed to single precision, if so, a slightly bigger performance improvement can be made. For this release, any speedups are of secondary priority and secondary interest, but if something can be improved for free, I'll jump at it. After today, there are 'zero risk' and bugfix changes only. In a way, 'testing/verification' is fun because it is nice to spend time listening to the recordings, but it also sucks because testing/verification requires serious concentration. I regret any implication that the release might be available sooner, because I very seriously expected to be starting the demos by now. The bugfixes and 'presence' correction took a little longer than expected. Changes have become more and more difficult because it is so important to avoid breaking the decoder/descrambler. Very sincerely, John Link to comment
John Dyson Posted January 28 Author Share Posted January 28 Avoiding the risk of 'TMI' (too much information), the project results might be delayed for approx 1wk. The delay depends on my available time before Monday, the large set of demos, and CPU/speed limitations to complete the demos. I have run out of time when considering unexpected effects of some medical issues. There was also a new 'unexpected decoder mode alternative' where last minute reviews could be completed in a day or so. That is, the 'unexpected new choice' might have been easily dealt with, except for the 'unrelated delay'. (The current 'slightly more dynamics' version does have complete demos, or almost so -- can upload the demos on request, but I really don't think that these will be the final version, especially considering the current 'first choice' and the new 'interesting alternative'.) * The 'interesting alternative' appeared as an idea about a 'distortion' found on the Aja album, and a suggestion by a reviewer that 'more similar' might indeed be 'more correct'. I tried this new 'interesting alternative', actually fairly easy to 'metaprogram', and found some 'goodness' in the result, minor improvement over the 'current 1st choice.' Current status: I am vacillatin, with bias, between two (now three) versions -- one version has completed demos. The evaluations require a lot of time for 'rest' between review sessions. Listening fatigue and increasing accomodation during each session really slows down the evaluations. Theoretically, a choice could be forced between the two 'good' candidates, but frustratingly, this attempt at being 'as correct as possible' has kept 'previous versions available' in case of finding unexpected bugs in the primary candidate. None of the evaluations have much to do with 'sounds good', but instead 'is there distortion'? There is a little less variance from FA in the new 'last minute' alternative, and might throw it away after an hour or so of review. * (current first choice) One version sounds more like the FA original, but more clean. * (plausible choice, hedged bet) The other version has more dynamics, still sounding similar to the FA original and more clean. * (possible new first choice) The new version is also slightly more like the FA original, just as clean as the rest. (less distortion in certain congested vocal chorus.) Like always, I am trying desperately to hear something that says: 'this is the correct result.' Until the new, third alternative, the 'most similar' WITHOUT the stronger dynamics seemed to be the most honest rendition. It will take a few hours of serious, patient review to choose between the 'new' and the previous 'most similar' without stronger dynamics are the best choice. The *unexpected* new 'candidate' has some positive attributes, and it is important to verify the previous best choice as 'best', or figure out why the new 'candidate' is better. Again, without the 'personal delay', the demo could likely have been available late tomorrow night as I had expected and publically suggested. If the medical delay ends up being a short distraction, then the release will be completed as reasonably soon as possible. (Frankly, I JUST WANT TO FINISH THIS -- IT IS WORKING WELL, I just need to find the *correct* answer, or *good enough* if there are no other alternatives.) SORRY, and WITH REGRETS (probably for approx 1wk.) If I do have time, it might be possible to produce some candidate snippets about the Aja distortion, but I am still trying to complete the release 'under the wire'. I'd rather focus on the release itself!!! John PS: if available sooner, I'll make the choice and send it out ASAP!!! Link to comment
John Dyson Posted February 1 Author Share Posted February 1 A new version with correction of a lot of 'little' bugs that added up to a lot. When comparing with the earlier V20E-RELTRY9, that earlier version is a total embarassment. It had a LOT of garble (corrected) and the stereo image smeared, lots of other little things -- again, adding up to a lot. The new version (coming soon) brings you up closer to the orchestra, much less 'diffuse' than the FA CD. At first, when finding all of the signals were totally stable, I was upset at the beginning of Symph #5 Allegro Con Brio, there was a big hole in the middle AT THE BEGINNING, until the instruments in the middle started appearing... Amazing!!! The original FA version had the instruments all smeared together into an ugly morass as would be experienced outside the back door of the audience chairs!!!😁 I am still iteratively working on a lot of variables, things like 'theoretically' 1.33 per the EIA component value, but instead had to be something like 1.33152... How do I know this very subtle change? I have to listen and have stable hearing. What was this 'theoretical' 1.33 value? -- the width factor for L/R<=>M/S conversion. It takes lots of testing to get the correct number. However, there is a STRONG tell, if sensitive to it -- the highs and lows become diffuse unless the value is correct. THIS DOES NOT HAPPEN IN NORMAL LINEAR SINGLE BAND SYSTEMS... On a single band system, most of what one would hear is 'too wide' or 'too narrow'. Also, the dynamics were previously over extended in some ways, fighting the other erroneous settings. The release is NOT ready yet, because I am trying to 'proof' the LF response below about 500Hz. Frankly, I am not happy yet, and trying to reverse engineer (read-the-mind-of the designer) the MF/LF processing. It is close to correct, but something is still wrong -- so no release yet. Some very interesting demos will be provided this weekend. The demos will definitely expose the evils of the FA processing almost all consumer recordings. The planned demos really are almost irrefutable proof of the 'decoder concept'. Initial reviews of real-world recordings might be a little different than expected, but instead better fit the reality of the studio. Other recordings still sound very similar between FA and decoded, so I'll only show one or two of those. Expect about 5-6 of diverse demos. The decoder design is TRULY a nightmare -- once the form of the decoder is 100% correct, even if the theoretical values are known, it could still take a month to find the actual correct real world values for the 'componentry'. The 'form' of the decoder has been correct since the final version of the descrambler had been added (the descrambler being first layer instead of last layer.) Since the component values had to start from scratch, the work has taken a lot more than just one month, instead more than a year. The demos could be produced now, but would be best to make sure the bass processing is the most correct possible. This work is tedious, but the bass should be close to correct by the weekend demos. John Link to comment
John Dyson Posted February 2 Author Share Posted February 2 Good News... Just got the decoder to nearly perfectly match some ancient stuff in my collection. The probability is very high for the promised demos this weekend. The demos will *not* be the full set of 95 because I simply will not have time before integration testing. After the several (probably 6-7) demos, I'll start working on a final test phase, looking for a few more nits. The more complete usual demos will come after final test. The bass is corrected and has a near perfect match against ancient references. Unfortunately, sometimes the FA processing does some things that will make the decoded version have an unexpected, but much much more correct sound. There have been so many 'little' bugs that I cannot list them all, but the previous source of diffuse sound and garble has been the incorrect L/R <=> M/S conversion factors. When the incorrect factors mix with the semi-nonlinear behavior of the descrambler, the sound ends up being similar to yet another FA scheme. Also, there have been bugs in the 'pilot' handling, but very happiily some non-released ABBA stuff on mp3 seems to produce plausible but technically incorrect results. Maintaining certain characteristics of the signal is critical, but mp3 and other lossy compression schemes don't maintain it. There aren't really too many surprises except that small, micro sized errors internal to the decoder might require a small tweak to the threshold (--coff=) setting. There are very very few recordings where the difference is noticeable, but when there might be a slight garble in the sound, just making a --coff=+ or - 0.00025dB change is all that is needed. I have never really acheived the goal so far, but the decoder is very very close if not working successfully now. Don't expect the demos before Sunday in the US -- I am doing fairly complete testing. John bogi 1 Link to comment
John Dyson Posted February 5 Author Share Posted February 5 Short delay -- the short list demos are in process (again.) I haven't liked the results so far, but the changes are only VERY minor. Actually, initially I do like the results, but my hearing is different than when making the selection. I want the sound to be correct, but I have been losing my hearing. There are about 5-6 3-way settings for the HF, and I am hearing 'fuzz' around the sound sometimes, then clear on other times. This kind of thing promotes fear in me. When I have more confidence in my ability to hear & review, I'll release the 'best choice'. There are more than several versions ready to go, each with minor variations in the HF. All are good prospects for being correct, just creating the versions in desperation. The results have been sounding different the hour or so between selection and listening to the demos. PS UPDATE (17minutes later): I think that after doing a big N-way comparison, I have the 'best' version so far. If my hearing is working correctly, most of the nits (error-tells) are very minor, except in the chosen version where I have heard none so far. So, the claim about a further delay might be premature, and a demo will likely be available in under 6Hrs (it takes time to do all of the logistics and final final final review.) So, there has been NOTHING wrong other than my inability to make a proper choice right now. There are no known objective measures available to me (or developed by me) for the descrambler behavior. Selection has to be done on the basis of a subjective A/B/C comparison... (Working full time on this beast until a short-demo release (perhaps 10 demos) is ready!!!) John Link to comment
John Dyson Posted February 5 Author Share Posted February 5 Thank you for your patient wait. This is the 'small set' of examples, not the monsterous set that are normally provided. Once things have settled down, and we are all sure that we have chosen the correct version, then the rest will be completed. Even these last steps, with a few choices, has been painful. My hearing has NOT been agreeable in the last few days. Anyways, the demos below are one of two sets that I could have chosen. My hearing is extremely unreliable, but I had to 'hold my nose' and choose. This version MIGHT have the sound of excessive 'stretching' (a wierd kind of grain.) If it does, then the 'other' version will be improved in that area. I chose this version because the dynamics seemed better and otherwise had ZERO tells beyond the possible stretching. LET ME KNOW IF YOU HEAR THIS 'STRETCH' SOUND, and if you do, I'll add the 'other' (V20L) version that DEFINITELY doesn't have the stretch for review. (The V20K seems to otherwise be 'tell free', but V20K might also have a fatal flaw... V20L corrects the fatal flaw if it exists, but the dynamics are more 'squished'.) I didn't want to delay this demo any longer, so here we go: * Please download first -- the dropbox player sometimes has the sound of 128k mp3... https://www.dropbox.com/scl/fo/zw5omel88g5x1ywh2ks8n/h?rlkey=egakjobeiyucr6xhvqf1cfj5b&dl=0 This has been tough, especially since so many earlier versions seemed to be complete. There has been even more things learned when creating this version (V20K-RELEASE.) I guess that the final version will be V21-RELEASE (I hope it can be available VERY soon -- big chance, right? 😉 John bogi 1 Link to comment
bogi Posted February 5 Share Posted February 5 I just did a comparison against originals as well as some middle October snippets and I am impressed by the improvement. My congrats! 🙂 i7 11850H + RTX A2000 Win11 HQPlayer ► Topping HS02 ► 2x iFi iSilencer ► SMSL D300 ► DIY headamp DHA1 ► HiFiMan HE-500 Link to comment
John Dyson Posted February 6 Author Share Posted February 6 6 hours ago, bogi said: I just did a comparison against originals as well as some middle October snippets and I am impressed by the improvement. My congrats! 🙂 Thanks for the feedback and thankfully I think we are finally zeroing-in on the 'solution'. ADD-ON: as typing this message, got feedback from a very trusted reviewer that the worried about 'stretch' problem just might be an issue. NO PROBLEM, will have the fix ASAP -- maybe delay a release by a day!!! GOOD FEEDBACK FROM SECRET REVIEWER!!! Current plans: (The schedule is currently indeterminant, but the '95 tracks' demos are planned to be complete by Wednesday, but lets give it Friday just in case.) At this point, we are right at the solution (the solution being the 'algorithm' for decoding.) I just did a few true-tweaks against some of the most difficult recordings and have started decoding my own collections of The Carpenters & the 8 ABBA albums. Of course, the purpose is two fold, one is for my own entertainment, but secondly I'll do a massive review of various tracks randomly. Other recordings have already been spot checked, with good success. After The Carpenters & 8 ABBA Albums, and given the liklihood of success, then the 95 example demo tracks and several other albums will be done. Assuming the likely positive outcome of all of these runs, it should be approx Wednesday (USA time.) I'll do the customary demo uploads in the Wednesday-Thursday timeframe, again ASSUMING all is okay. Currently, there are zero failures for decoding, with a few moderately mediocre results (the SuperTrouper track used to produce miserable results, now just mediocre.) There might be some minor problems with the recordings being 'over stretched', but with the unlikely need for a solution, an answer is immediately available. (Over stretching causes a graininess in the sound, but given what I know now, if it needs fixing, the correction is almost trivial.) Right now, I cannot hear well enough to hear the stretching, but all other aspects of the decoding results appear to be satisfactory. After the demo runs are complete, and once I am happy -- then I'll have to work on the Windows version of the decoder. I AM STILL OPEN TO CRITICISMS ABOUT THE SOUND QUALITY, AND WILL ADDRESS THEM IMMEDIATELY. I doubt anything beyond the 'stretch correction' or 'tweaking' will ever be needed from now on. John Link to comment
John Dyson Posted February 6 Author Share Posted February 6 STATUS: I'll be restarting the first (short) demos a little later today. Got some feedback by a very kind reviewer that there was indeed a 'stretch' in the sound, so the 'solution' was readied. After doing a complete set of testing/reviews, it appeared that *something* that had been worrisome about the bass and also a problem with decoding the 1992 ABBA Gold albums persisted. Even more testing/reviews and enabling previously disabled internal EQs, the quality & accurate detail improved markedly, with seemingly even less distortion. The feedback about the 'stretch' in the sound enabled further improvements, now pretty much ready. Still doing a few more verification tests and focusing on avoiding unpleasant surprises. These 'further' improvements were mostly disabled code. It seemed like the disabled code was needed, but the disabled code make the sound worse, not better. Good news: original recent conceptual understanding was found to be correct, and more of the expected code segments are now available to improve and maintain sound quality. There were disabled phase reversals, which actually should have been enabled, but other 'bugs' made the results worse. The new demos and subsequent actual full demo release will not be hurried, but a significant delay is not really expected. Very little code is now being written, mostly it is enabling existing code and finding more accurate/more correct control parameters. Will message in a few days with further updates, unless something really major is found. John Link to comment
John Dyson Posted February 7 Author Share Posted February 7 AIEEE!!! Looks like I made a major architectural error in the decoder, but it is moderately easily correctable (a few days of work.) Here is the problem: I have never really been happy with the decoding results, they had always had a bit of hiss modulation and the dynamics wasn't quite 'just right'. I have been adjusting the decoder to perfection, but something -- WRONG!!! Not bad, just not 'complete'. Answer: The decoder appears to need a final M/S DolbyA layer. Hmmmm.... There have been some real test improvements. I greatly regret and apologize for the bug. Sadly, adding this layer will slow down the decoder by about 10%. With the vast improvement, it might be possible to run in a slower mode -- just don't know what is going on yet. This addition of the DolbyA decoding M+S layer might still be wrong, but the initial tests are amazing (my bad/unreliable hearing is the only variable here.) I'll be asking a few reviewers for their opinon on the results, and if I become more confident, will make some demos available publically so as to keep everyone still interested completely involved. It might have been easier to consider adding this layer as a bugfix and not making a big deal out of it. I think it is better be transparent and show how easy it is to make stupid errors on something as complicated as the FA decoder. This thing is a TOTAL BEAST. John Link to comment
John Dyson Posted February 8 Author Share Posted February 8 Sort of good news.... My own initial reviews show that the new final DolbyA decoding layer is very important. The results move from 'improvement' to very noticeable improvement. The decoder is now 'wired' to be able to operate in the 'new' mode and 'old' mode. By the time there is a release, the '--fa' switch will make the correct choice, and the alternative mode will be a 'hidden' '--fs' switch. I will not be offering it (demos/decoder) to the reviewers and the public until the errors appear to happen a little less often. Running a DolbyA decoding layer with it's profound expansion and narrow range of activity does require the settings in the descrambler and earlier FA layers to be precise. The fact that the final DA layer doesn't go totally nuts is a testiment to the FA decoder doing something generally compliant. There are some errors that happen perhaps in 1/20 recordings (just a wild estimate), instead it would be best to decrease this noticeable error rate to 1/100. (Some errors are in the recordings though.) Errors might include an unexpected slight shift from right to left (vice versa). Whenever this problem does happen, it is important to make sure that the shift isn't a bug in the recording itself. MOST of the bugs are in the recordings, but still too many bugs appear to happen in the decoder, made more noticeable by the final layer that decodes DolbyA material. ADD-ON: just double checked a current internal set of running results for demos and the error rate is already better than 1/20 and just might be 1/100, just not sure yet. There are a few subjective improvements needed (errors are multiplied by the new DA layer), so just a little work is needed to slightly soften the output. That is, the HF processing before the DA layer is just a little too strong. OTHERWISE, the current results are mega-encouraging!!! Finding this d*mned problem will, of course, cause a bit of a delay. It feels like the 'finish' is a continual tease. If it seems like the project is a 'forever' one, just think about how it feels to me!!!! As a personal matter, I WANT THIS TO BE COMPLETE. My own integrity requires that the decoding algorithms are at least plausibly complete with no known path for improving or correcting the architecture. A new 'path' beyond the state as of yesterday needs to be traversed. FRUSTRATED, but a bit relieved that the quality is very very noticeably improving. John Link to comment
wilkiamieva Posted February 8 Share Posted February 8 Hi, John! I was reading about CD pre-emphasis and de-emphasis today, and I had an idea... Could the differences in performance that you find in your demos be caused by a lack of de-emphasis on audio extracted from old CDs? Just my two cents. Keep the good work! Wilki Link to comment
John Dyson Posted February 8 Author Share Posted February 8 12 minutes ago, wilkiamieva said: Hi, John! I was reading about CD pre-emphasis and de-emphasis today, and I had an idea... Could the differences in performance that you find in your demos be caused by a lack of de-emphasis on audio extracted from old CDs? Just my two cents. Keep the good work! Wilki Thanks!!! It is always a good idea to consider ideas like yours (the pre-emphasis thing) every time there is a problem. Also, it is usually a good idea to consider a simpler solution rather than the complex one like the new DolbyA layer. Next time we run into a serious problem, I'll look at the pre-emphasis. It has probably been 5yrs since I looked at the pre-emphasis matter, and before the next release, I'll look at it again. It is time to revisit pre-emphsis. I do have a suspicion that by far most of the questionable decoding results have been remedied by this DolbyA layer. It can correct a myriad of problems, however once the new layer of DolbyA like processing is added, we (myself, helpful potential users, reviewers) will certainly look for new problems. Perhaps the pre-emphasis issues might become more obvious once the additional layer has been fully integrated. Adding the new DolbyA-like processing layer was one of the last things that I would have preferred to do. This new change was considered because certain recordings very obviously require the additional layer. On a lark, I tried the additional layer on recordings that I would never have considered to need it. Wow, I was surprised. It appears that the new layer might not be optional. If the new layer needs to be adopted, the architecture will look something like this: input -> descrambler -> 3 layers of (pairs of M/S and L/R DA in sequence) -> one final layer of L/R DA -> output (M/S is mid/side, L/R is left/right. M/S is an alternative form of stereo.) This new scheme requires a total of 7 DA layers -- I did not want to add the additional layer, at first, not feeling totally comfortable with it. Likewise, during the next attempts to do a release, I'll simulate recordings with incorrect pre-emphasis and listen to the results. It is possible that yet another error syndrome might be found, and I will DEFINITELY give it a look-see. THANK YOU!!! John Link to comment
John Dyson Posted February 8 Author Share Posted February 8 The new version of the decoder with the DA layers built-in instead of using FA layers only -- ALMOST ASTOUNDING... It is still a work in progress, but I expect zero change WRT noise reduction on this ancient Brubeck recording. There is still a very subtle amount of noise pumping, but this NR is better than I had ever expected... Also, the cymbals will naturally be 'more tight', perhaps slightly lower level, but that is a side-effect of whatever expansion being done. RAW consumer recording: 04-Three To Get Ready-FARAW-SNIP.flac DECODED version, still a minor amount of work to do before release: 04-Three To Get Ready-V20MEXP10-SNIP.flac * There is still a phase problem, yet one of the myriad of 'minor' things that need resolution... On a spectrogram, the NR is also pretty darned good. Of course, as mentioned above, low level signals at high freqs might have a small amount of attenuation, but should be more an adjustment to being correct rather than a true lossage. It will still require another day to do the extreme exhaustive testing that must be done. Even after *I* do testing, there might still be something that I miss -- I am full of expectation bias and other development biases. No matter about biases, this should show pretty credible progress. This development has been painfully slow, probably other people could have done the development more quickly, but progress is being made!!! John Link to comment
John Dyson Posted February 9 Author Share Posted February 9 Follow-up on the really good NR example... There was feedback that the piano didn't sound quite right in the demo... That observation is correct, partially caused by the phase problem that was previously mentioned as a known defect. There are monsterous numbers of variables that need to be coordinated. The piano being off by some amount is undesirable, but is part of the 'phase problem'. Just this minute, targeted 'short demos' are being run. Once the results seem to be close enough for me, then I'll make them available. The general sound is more than noticeably better than when the previous demo was created. 1) phase problem in midrange... 2) Misguided dynamics setting > 9kHz 3) erroneous dynamics parameter (type-o type thing.) John Link to comment
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now