A toast to PGGB, a heady brew of math and magic

[austinpop]

1 hour ago, ted_b said:

I am please to announce that, starting Aug 6, NativeDSD will begin selling the Patricia Barber Clique album in 32/352.8k, the original recording format.  Zaphod, Rajiv and others asked us to do this and we obliged.  We'll see how the sales go.  I will push for other labels/albums as we move on.  👍

 

Note: also, we will sell the album in DSD1024 (the first stereo DSD1024 I am aware of) and 32 bit DXD multichannel

That’s great news Ted, especially for PGGB users. 32-bit DXD files retain more of the precision from the original master, giving more information for PGGB to work with.

 

I hope PGGBers in particular, and ASers in general, support and buy this format.

[austinpop]

4 minutes ago, Zaphod Beeblebrox said:

It turns out it is a fast internal SSD drive and the only partition is the OS partition and it is a 4TB SSD with 2.6Tb of free space, so really bummed why it will not grow beyond about 120GB. Perhaps some mac gurus can chime in.

Ugh. I know people love to hate on Windows, but at least on this issue, I’ll take the explicit allocation of paging space in Windows any day.

 

Don’t get me wrong, my laptop is an MBP, so I like MacOS too.

[austinpop]

3 hours ago, happybob said:

But I am currently running it on a M1 MacBook Pro (>1TB of free SSD space and 16GB of RAM)

 

3 hours ago, happybob said:

the dreaded "out of application memory" error and then often a PGGB crash

 

I'm not clear why MacOS, with >1TB of free space (presumably on you boot drive?) would cause the app to throw an OOM. Perhaps as @kennyb123 says, it could be an artifact of Intel emulation on M1.

 

Calling MacOS experts...

[austinpop]

Very useful MacOS suggestions and observations, @ray-dude and @kennyb123.

 

"To OOM it may concern." 🙂

[austinpop]

29 minutes ago, davide256 said:

Trying to figure out if the Denafrips Pontus/Venus/Terminator with dual AES/EBU inputs are an alternative to the dual BNC Chord solution. Anyone use this?

Struggling to find a description/limitations.

 

The reason so many of us Chord users are using a USB to dual-SPDIF bridge is to bypass the Amanero USB controller, in favor of a lower-latency input, dual-SPDIF. But you're lucky your DAC has an XMOS2 controller, that is known to not have the latency issues of Amanero, so I would consider going USB direct to the DAC. I understand the Gaia/Iris DDCs may have other benefits, but at that point you would be weighing between the benefits of:

1. using a DDC with native files (no PGGB)
2. going USB direct using 32FS PGGB files (for Denafrips DACs).

As for DACs with "dual" SPDIF or AES inputs, I'm not aware of many. There's Chord, with the dual BNC SPDIF (16FS), dCS with their dual AES (but only goes to 8FS), and after that I'm drawing a blank. Do the Denafrips (or any other?) DACs support this "dual" mode on their SPDIF or AES inputs? Anyone?

 

[austinpop]

1 hour ago, Egill23 said:

The Denafrips Terminator has dual AES/EBU input. From the manual:

Dual AES/EBU Input

1. Press the Mute button once to enter configuration mode

2. Press the INPUT+ momentarily, AES 1, AES 2 LED will turn on/off

-AES1 On = Dual AES/EBU Input Enabled

-AES2 Off = Dual AES/EBU Input Disabled

3. Wait for 10s

4. DAC back in operational mode

 

Interesting. What are some examples of dual-AES sources for the Terminator? And what is the max rate the Terminator can accept in dual-AES mode?

 

I thought these dual data modes were proprietary, like the dCS example Steve Z mentioned above. I'm pretty sure Chord MScaler also only works with Chord DACs in dual-data mode.

[austinpop]

5 hours ago, chrille said:

And how could I maximise the number of taps with  an Opera recording that has got maybe 50-60 different "tracks" some lasting only 20-30 seconds and others 20 minutes or more and the complete work lasting 2-3 hours or even  3-4 hours as in some Wagner Operas?

Is PGGG capable of processing such long works as Operas and still maximise taps?

 

Chrille,

 

Many choral pieces are indexed in this way, even though the music is essentially continuous. IF this is the case, and you are willing to give up (or at least forego) this fine-grained indexing, there is a way in PGGB (look for combine.json in the manual) to combine multiple short tracks into one longer track that can get the benefit of a longer filter.

 

I have posted about this previously here: https://audiophilestyle.com/forums/topic/62699-a-toast-to-pggb-a-heady-brew-of-math-and-magic/?do=findComment&comment=1132878

[austinpop]

17 hours ago, happybob said:

1. Highend listening: PGGB files get played back directly (not via Roon) to an SRC-DX via USB and then out to an MScaler via dual SPDIF cables. The MScaler output goes to a Dave DAC via another set of dual SPDIF cables. In this scenario only 24 bits get sent to the MScaler. The MScaler then (unavoidably unfortunately) does a 3dB level adjust AND more importantly does its own noise shaping (can't disable this). The MScaler out to Dave is also 24 bits. So for this mode, either 32 bit or 24 bit "no noise shaping" is optimal, but does it matter which?

 

I understand the convenience factor with this configuration, but before you settle on this long term, you may consider doing your own listening tests to confirm how much you lose in SQ (subjectively, to your ears, in your system) between the above (call it A), and an optimal configuration B, where you connect to an SRC-DX via USB and then straight out to the DAVE via dual SPDIF cables.

 

In A, you'd be feeding non noise-shaped 24-bit data to the chain, while in B, you'd be feeding noise-shaped 24-bit data.

 

In my past listening tests, when ZB first introduced noise shaping in development, the increase in SQ from noise shaping over standard rounding/truncation (only to 32 bit — this was pre- SRC-DX, mind you) was quite substantial. Truncation effects on 24-bit would be even more severe.

 

Still, ultimately, it's your system and your ears, so if the SQ delta between A and B isn't too large, then the convenience factor of A may be worthwhile.

[austinpop]

It's a definite uptick in SQ. I've redone all my 1FS albums to 24/16FS using v2.1.05. 

 

Luckily, PGGB is blazingly fast for 1FS files, so it goes relatively quickly.

[austinpop]

@HeeBroG

 

Just as @Zaphod Beeblebrox explained above, my 1FS albums process at rates between 2.5x to 3.5x, depending on track lengths.

[austinpop]

For those looking for more clarity on how (and why) to use the combine.json feature, here's an example album I just ran into. 

 

https://www.qobuz.com/us-en/album/gustav-mahler-symphony-no7-chicago-symphony-orchestra-claudio-abbado/0002894455132?qref=dpa

 

While this symphony has 5 actual movements, the CD is indexed into 21 "tracks." To coalesce these back into the 5 actual movements, thus allowing longer tap length filters to be used by PGGB, you would create a combine.json file with a single line (below) and place it in the album folder prior to processing with PGGB.

 

["1:8", "9:12", "13:15", "16:17", "18:21"]

[austinpop]

2 hours ago, kelvinwsy said:

Of course .. All observations and comments are totally Dac and system dependent' zmy ears and my tastes as well as my gear are Mine!!

No slight on what PGGB hopes to achieve but online upsampling with especially aphrodising filters is much more to my tastes..

YMMV As usual

 

Interesting comments. It must be a DAC-dependent thing. 

 

I have no experience with the Micro iDSD Black Label, but I have compared PGGBed 24/16FS files on the iFI Pro iDSD. The comparison was:

• native files, using the built-in GTO (Gibbs Transient Optimized) filter
• PGGBed 24/16FS file, which automatically triggers the bit-perfect (BP) mode in the DAC.

For this comparison, the benefit of PGGB was large, and obvious. This was on a friend's setup, and he doesn't have HQP, so we never compared to HQP.

 

 

[austinpop]

2 hours ago, Kalpesh said:

However I have tested with quite a success the theory that PGGB could be a nice pre-stage from 44 to 352 64 bits f, no apodizing, before feeding HQP that does the eQ and the last mile up to 1.4M

 

May I ask: what DAC are these findings on? My experience is nothing like yours, but then we may be on totally different DACs. I'm on a Chord DAVE.

[austinpop]

21 hours ago, mcewan71 said:

1. So I get how this works in theory with the taps etc. I am just wondering how this actually happens - it’s kind of ‘breaking apart’ the PCM file then restructuring it from the pieces, right? I guess I have the experience of understanding that with digital audio files, you cannot put in what was not there in the first place (e.g. creating a lossless file from a lossy one would be a simple example). So - without reference to the information on the website (I know where to find it, and I think I understand the technical basics), could someone please humour me and translate this into layman’s terms? I guess I need to completely get my head round how I can feed in a PCM file of whatever bit depth and sample rate - and get a much better sounding file the other end. 

 

Wow, that's a tough question to answer. On the scale of "what is the answer to life, the universe, and everything?" 🙂 Short answer: 42. Longer answer: you really should look at @Zaphod Beeblebrox's FAQ at https://www.remastero.com/faq.html It's an easy read, not very long, and meant for laypeople, as you asked. At a minimum, please look at the section titled: What are 'Taps' and how many are too many?

 

21 hours ago, mcewan71 said:

2. It’s probably fair to say I am definitely along the budget end of the scale when it comes to the DACs I have. Of course, this will change over time and circumstances, but my budget DACs max out at 24/96 just now. As much as I would love to buy a Dave or even a modest step up from what I have now, it’s not within my reach currently. For my listening rooms, I use iFi Zen DACs and for headphone listening I use Audioquest Dragonflies, both red and cobalt (told you they were budget!). Point is, I enjoyed what I heard with the PGGB files using these means. As I said, I will invest in higher spec equipment over time. I just read of what seems like insane sample rates folks’ DACs handle in here, and find myself wondering if it would be pointless getting a license given my current limitations? 

 

I would suspect the benefit of upsampling from 44.1/48 kHz to 88.2/96 kHz (a factor of 2) to match your DAC would be modest. However, there's no substitute for experimentation, and that's what the free trial is for. If you like the uplift in SQ with your current DAC, and you have a plan to upgrade DACs over time, then a license could make sense for you. You should listen and decide for yourself.

[austinpop]

Ok folks, enough with the bickering. You don’t like PGGB, that’s fine, move on. Nobody said it’s for everyone and all DACs. 

[austinpop]

38 minutes ago, edwardsean said:

I don’t know if this was addressed before, but do those of you using Dave, engage its HF filter with PGGB 16fs files?

 

I had it turned off as I assumed it wasn’t necessary since PGGB has its own HF filter. However, I’ve been experimenting and enabling it has a positive effect. To my ears, it seems to be cleaning up some out of band noise coming from somewhere. 

 

I personally prefer it Off, while setting PGGB's HF Noise Filter = Minimal. 

 

YMMV, of course.

[austinpop]

49 minutes ago, GryphonGuy said:

However, after reading that @austinpop uses the "Minimal" or "more airy" button, I decided to try it.

 

Woh! very effective on 44.1kHz material even only upsampled to 176.4kHz.

 

So is that comment about being ineffective on less than 88.2kHz material a left-over from the heady development days and now after revisions the HF Noise Filter is effective at all original track sample rates?

 

GG,

 

Based on my understanding - defer to @Zaphod Beeblebrox for a definitive answer - there should be no effect of the HF Noise Filter setting for 44.1 tracks. So I don't know why you're hearing a difference.

 

Did you change anything else at the same time as you switched from Moderate to Minimal? PGGB versions, for example?

 

For the record, here is my current preferred configuration:

[austinpop]

5 hours ago, GryphonGuy said:

The only difference in my setup now is that I use "Auto(2048)" maximum taps (Million).

 

It must be something else, but would be good to get to solve the mystery.

 

What was it before you set it to 2048?

 

If you have a specific track on which you hear this improvement, can you post the full file name of the "before" and "after" tracks? 

[austinpop]

10 hours ago, LowOrbit said:

128bit precision brings new depth, more harmonic richness, more tonal body, gret acoustic realism to instruments and the space they were recorded in. If PGGB relaxes complex usic and lets you dig into it, 128bit spreads it all out, teases out more goodness and lets more enjoyment into your ears.

 

I'm in complete agreement with you. This is very much my impression too. The key differences with 128-bit, and even more so with 192-bit, are liveliness, excitement, richness, and realism.

 

I had known conceptually that performing DSP computation at much higher precision had a positive impact on SQ. But doing this listening comparison side-by-side, with tracks processed by PGGB at 64-bit, 128-bit, and 192-bit, the differences are really quite startling! You're starting with the same 16/44.1 file (say), ending with files of the same 24/ or 32/705.6 resolution, but what a difference the precision of the intervening math makes! Of course, a great deal depends on the quality of the noise shaper that takes the 128- or 192-bit floating point results, and renders them as 24- or 32-bit integers. All credit to ZB for this wizardry!

 

 

10 hours ago, LowOrbit said:

In short, I am loving the new layers of reveal, the added tonal richness and harmonic life (pianos and guitars come alive) that this new update brings. 

 

And again: very well said, and couldn't agree more.

[austinpop]

The SQ benefits of 128-bit (and higher) precision in PGGB-AP can be heard across a variety of players.

 

Yes, some of us have used HQP in the way ZB described, but you get the same  benefit with bit-perfect players like TAS, Stylus, and even Roon. The key is to disengage all DSP in the player.

[austinpop]

10 hours ago, Zaphod Beeblebrox said:

There have been numerous requests to improve memory performance of PGGB, especially to allow high precision computing without the need to invest in a lot of RAM or in Server grade hardware. We have been working on addressing this while improving on the algorithms too. Many thanks to those who have been patiently waiting and many thanks to the beta testers for their feedback and encouragement, we have been working on this for the past 9 months and constantly improving. There was even an intermediate version (v4) which we skipped over to bring v5.

 

PGGB 256 Infinity and Beyond Edition

I am happy to announce PGGB 256 v5.0.42. PGGB-AP is now PGGB 256. PGGB 256 uses completely revamped algorithms, but still follows the same fundamental principle of getting as close to perfect reconstruction as possible. The new algorithms are optimized to maximize reconstruction accuracy to near theoretical limits, and it improves over what PGGB-AP did.

• PGGB 256 uses highly optimized custom built multi precision math libraries. PGGB 256 is now faster and more memory efficient than ever before.
• PGGB 256 uses a minimum of 128bit precision for resampling (even for 64bit processing), one can optionally use up to 256bit precision for all of the processing.
• The configuration menu is much simplified and there is an automatic mode that now chooses the optimal settings based on each track. More details can be found here.
• PGGB 256 does away with having to choose taps, does not use windowed sinc function and is non-apodizing.
• PGGB 256 now has a 'XDMS Mode' under the hidden menu to split output files into 1.25GB parts to aid play back in Taiko Audio's XDMS.
• PGGB 256 now supports 64bit output that is noise shaped, this is useful if you wish to process these tracks further.

 

Licensing options have changed

There are now three types of licensing in addition to the free trial. More information can be found here.

1. Trial licenses: They have been simplified, PGGB 256 will automatically start in trial mode and one need not request a trial license separately. As before it limits processing to 5 tracks at a time for 64 bits and 2 tracks at a time at 128 or 256 bits.
2. Perpetual (64): Existing PGGB-AP perpetual license holders are automatically upgraded to 'Perpetual (64)' license for PGGB 256.
   1. You can run previous version (v3 or older) side by side with PGGB 256.
   2. PGGB 256 will run without any limitations when set to 64bit processing. Even when set at 64bit processing the resampling is done at 128bits and one also gets the full benefits of the new revamped algorithms.
   3. When set to 128 or 256bit, processing is limited to 2 tracks at a time (one can restart to process couple of tracks again)
3. Perpetual (256): This allows end-to-end processing at 256 bits without any limitations.
4. Upgrade to (256): Existing PGGB-AP perpetual license holders or 'Perpetual (64)' license holders can upgrade to 'Perpetual (256)' license.

Where is my cheese

If you have been using PGGB-AP, PGGB 256 uses a different configuration file for settings. This allows PGGB 256 to coexists with previous versions. It also means, you have to setup PGGB 256 again with your default settings (which it will remember going forward).
Some of the previous settings are no longer available as they are not relevant:

1. Most notably, no more taps: PGGB 256 does away with the concept of taps. The new algorithms do not limit the quality of output based on the available resources, it uses an adaptive processing pipeline to achieve the same reconstruction quality with the available resources*. 
2. Transparency and presentation options are gone: With windowed sinc function, there is always a choicebetween time domain and frequency domain performance, and these controls provided the parameters to tweak windowing. PGGB 256 does not use windowed sinc functions and these compromises are no longer needed.
3. Apodizing check box: The check box is gone as PGGB 256 is non apodizing.
4. Hidden menu option DSD throttling: This option is no longer needed and if TDP is an issue, it can be controlled by setting the number of workers.
5. Hidden menu option Disk Caching: This option is gone because PGGB 256 is a lot more memory efficient. 

*PGGB 256 still needs enough memory available to hold input and output buffers at the desired precision, the processing however is designed to trade speed to fit into memory. The required memory proportional to the length of the track.

 

Having been involved closely with ZB in his creation of PGGB, I know how much it has evolved and improved with each iteration. As existing PGGB users know, PGGB-AP at 192-bit resolution was a major step forward in improving SQ over the original PGGB at 64-bit precision.  Yet, I consider PGGB-256 to be an even more significant upgrade than PGGB-AP.  Some of you may be old enough to remember the old Miller Lite commercial: "Tastes Great. Less Filling." With PGGB-256, it's more a case of "Tastes better. Less filling!" PGGB-256 gives you: 

• another dramatic uptick in sound quality, but equally importantly,
• a dramatic improvement in performance, where even the most pathological tracks can process comfortably on a consumer desktop.

I've already described my SQ impressions of PGGB up-thread, so I won't repeat myself again. PGGB-256 just gives you more of the goodness: coherency, transparency, density, soundstage size, and layering. Rather than going on my or anyone else's descriptions, the best thing would be to give it a try on your favorite tracks, and see what you think.

 

For existing users, even if you don't upgrade to the p256 license, you'll definitely hear a substantial uptick in SQ. Should you upgrade to 256-bit? Try a few tracks and have a listen. What I can tell you is that for me, it was a no brainer, and I sent in my upgrade fee (yes -- I paid the same fee as everyone else) immediately. It also says something that as far as I know, every single one of ZB's beta testers opted to upgrade as well. The benefits, once heard, are hard to unhear. Such is audiophila!

 

A final word on performance. It is rare when a hobby intersects with one's professional life. In my case, performance engineering is my profession, so helping ZB tune the performance of PGGB-256 (and earlier versions) was satisfying in a different way than work projects. I'll admit it sometimes felt surreal to be analyzing PGGB CPU/memory/disk profiles, knowing that this was helping my enjoyment of music. A very rewarding experience!

[austinpop]

16 hours ago, gtl said:

Has the hardware requirements been lowered for MacOS with the latest PGGB 256? I would get a M1 Pro MacBook or a M2 Pro Macbook with 16gb of ram along with PGGB 256. Mostly listen to redbook or 24bit PCM with the occasional DSD64. Can the files be processed?

 

I would also suggest that running PGGB on a laptop, even with adequate memory should only be considered for occasional use. When running PGGB on a bunch of albums, you will have sustained periods of high resource (CPU, memory, disk) utilization for hours, so a system with good cooling is essential. Laptops in general are designed for interactive use, not for sustained workloads, which can cause your fans to run loud, and heat to build up, possibly resulting in CPU throttling. I'm not saying every machine will do that, but just as a general guideline.

 

A desktop computer with a good CPU cooler, and a case with good airflow, is a much more sensible option.

[austinpop]

To give an indication of just how much faster PGGB-256 v5 is, compared to PGGB-AP v3, I went back and found some runs from v3, which I compared to my latest runs on v5. The speedup results are impressive! To summarize:

• On these 3 examples, v5 completed PGGB resampling 4.5x to 6.5x faster,
• While processing at a higher 256-bit precision compared to 192-bit in v3, for the PCM albums, and
• 256-bit precision compared to 128-bit in v3, for the DSD album. Why only 128-bit on v3? My machine had insufficient memory to run v3 at 192-bit precision for DSD inputs.
• I've yet to encounter a track that caused an out-of-memory error, or resulted in severe paging on v5 running PGGB at the full 256-bit precision on my 128GB RAM machine, even on my torture tracks that exceed 30 min duration in DSD or DXD resolution. On v3, such tracks could not be processed on my machine until I dropped down to 128- or even 64-bit precision, and even then they would cause severe paging in the system.

 

Timing Comparisons v3 vs v5

 

 

Native sample rate 24/96

Track duration range: 4-10 mins

 

                                    v3 @ p192   v5 @ p256    Speedup

Time to process album     11h, 47m     2h, 38m        4.5x

 

 

 

 

Native resolution: 16/44.1

Track length range:1-6 mins

 

                                    v3 @ p192   v5 @ p256    Speedup

Time to process disc 2       7h, 18m     1h, 22m        5.3x

 

 

 

 

Native resolution: DSD64

Track length (disc 1): 24 mins

 

                                    v3 @ p128   v5 @ p256    Speedup

Time to process Disc 1          15h, 47m          2h, 26m         6.5x

 

 

[austinpop]

Fixed the images in previous post.

[austinpop]

1 hour ago, Zaphod Beeblebrox said:

I have some more information on the memory requirements. Compared to both the original PGGB and PGGB-AP, even at a precision of 256 bits, PGGB 256 is a lot more memory efficient. 

 

PGGB 256 uses adaptive algorithms that do not compromise on the quality of reconstruction to fit in available RAM (previously one could do this by setting taps). Currently the reconstruction quality at a given precision does not change, instead PGGB 256 adapts the processing pipeline to fit in memory and trades speed to achieve this (i.e., runs slower, to make sure the data being processed fits in memory).

 

What this means is one could do a lot more with 16GB RAM and PGGB 256 than previously possible. But there is still a limit to how much can be made it fit in RAM. At the minimum one needs enough RAM to hold the input data and upsampled data. So, this memory requirement depends on a few factors:

• Length of the track you are processing
• Input rate of the track (CD vs Hi-Res vs DSD etc.)
• Output rate (which depends on your DAC and settings, if you are upsampling to 705/768khz or 352.384kHz)
• Precision (64, 128 or 256 bit)

 

It is possible to estimate the memory requirements, so that is what I did, I simulated what is possible with 16GB of RAM (these are still estimates and should be considered approximate, it is best to try on a PC or laptop to confirm). I have also attached a pdf. Below is a summary of what to expect with 16GB.

 

Macs do not allow user configuration of swap files but typically allow swap file to grow to at least half of available RAM, so on a Mac, the maximum I will be comfortable with is a max processing memory of 24GB on a Mac with 16GB RAM. Below all the cells marked with orange imply you will hit a limit on a 16GB Mac.

 

• If you are upsampling to 8fS (353/384kHz), you should be fine with most tracks within 12minutes of length even at 256 bit precision.
• If you are upsampling to 16fS (705/768kHz), you should be fine with most tracks within 12minutes of length up to 128bit precision. 
• If you are upsampling to 16fS (705/768kHz), you should be fine with most tracks within 6minutes of length up to 256bit precision.

 

For Windows, with 128GB of VM allocated, up to 12minute track at 256 bit precision would not be an issue (it would page for 16fS upsampling and would be slow). Here too 32GB RAM will be good to have.

 

I also have a datapoint. Somone with the new Mac 16" M2 pro with 16GB RAM did this album at 256 bit precision to 16fS in 2 hours, 12 minutes. The same album took on a PC with 16 core Threadripper 1950x and 128gb ddr4 1 hour and 2 minutes. 

 

Compare memory 256.pdf 63.73 kB · 3 downloads

 

Just want to clarify, in case it wasn't clear, that all the numbers in the table are virtual memory. 

 

On WIndows, the available virtual memory is approximately (Physical RAM + paging file(s) size). So, even if you don't have the required physical RAM, your runs will not fail, if the total amount of virtual memory is sufficient. The only downside is that the bigger the difference between the memory required and the actual RAM, the more your system will be paging, i.e. writing memory pages to and from disk. This is why a fast SSD is crucial for your paging file.