A toast to PGGB, a heady brew of math and magic

[dmance]

Regarding DSD over DOP via SRC-DX. SRC-DX was only specified to support DOP DSD128 ...as this is a single SPDIF stream and the DOP mechanism is defined.  The CT7601 chip used does have the internal DOP support up to DSD256  but the AES dual-date-date mode to support DBNC does not define DoP.  If it does work ...its ad-hoc and fortunate. I need to ask a SRC-DX/Chord DAVE owner to try this out. *thanks*

[Miska]

7 minutes ago, dmance said:

Regarding DSD over DOP via SRC-DX. SRC-DX was only specified to support DOP DSD128 ...as this is a single SPDIF stream and the DOP mechanism is defined.  The CT7601 chip used does have the internal DOP support up to DSD256  but the AES dual-date-date mode to support DBNC does not define DoP.  If it does work ...its ad-hoc and fortunate. I need to ask a SRC-DX/Chord DAVE owner to try this out. *thanks*

 

There is a specific dual-wire DoP specification, amended to the original DoP standard. I have it working fine to the good old Mytek Stereo192-DSD DAC over Firewire from macOS to do DSD128 (4 channels of 176.4k).

 

[Zaphod Beeblebrox]

17 hours ago, dmance said:

I need to ask a SRC-DX/Chord DAVE owner to try this out. *thanks*

@HeeBroG Couple of people have tried with  Chord DAVE and confirmed DSD256 does not work over DoP with SRC-DX.

[sigmadelta]

22 hours ago, Zaphod Beeblebrox said:

@HeeBroG Couple of people have tried with  Chord DAVE and confirmed DSD256 does not work over DoP with SRC-DX.

 

Can they confirm please DSD128 over DoP with SRC-DX?

 

With their Chord Dave

 

[Schafheide]

I am very much an ab initio at this gargle blasting thing.

Given that, depending on your PC and your chosen settings, the gargle blasting can sometimes take hours (and hours) to complete.  I wonder how hard it would be for the ap to calculate the approx completion time for the target folder and then display a moving timeline (similar to file copying etc) ??

[Zaphod Beeblebrox]

19 minutes ago, Schafheide said:

I am very much an ab initio at this gargle blasting thing.

Given that, depending on your PC and your chosen settings, the gargle blasting can sometimes take hours (and hours) to complete.  I wonder how hard it would be for the ap to calculate the approx completion time for the target folder and then display a moving timeline (similar to file copying etc) ??

The completion time depends on a number of factors, it is not a simple function of the track length; if it were, it will be easier to calculate. I will have to think about it, it is a good idea.

 

[rayon]

I also have quality-of-life thoughts related to stopping and starting the processing:

1. "Immediate pause/stop". Currently when clicking pause, it still spends a lot of time processing the current file, sometimes even 2h. I basically have to plan my listening sessions ahead of time as listening is stopping occasionally when PGGB is working in the background. In my dream state I could just click on Pause and PGGB would stop the processing immediately (releasing the computer for other use in 1 minute or so), release RAM and write cache into some temporary file.
2. Save before pause/stop (if the above is not feasible): When using Pause -functionality, it pauses right before it safes the file it just processed. That last step of just saving the file takes usually like 10 seconds or so and compared to time the processing takes, it's basically nothing. Especially in the beginning I ended up situations in which I paused it, then stopped and then realized that even if it finished, it actually never saved the result and I had to do that heavy calculation again. I would suggest that when pausing during the processing, it also saves the file and then pauses. Same for stop. I'd guess that this shouldn't be too many lines of code, but would make it behave much more logically.
3. Time window when it's allowed to start new work. I'd guess that "immediate pause" is hard to implement as it's relying on mathlab under the hood, but if user could give PGGB time window for when it can start new work, I could put there something like 24-06 as I basically newer listen to music during the night and even if it started to process something right before 6AM, it would be done before 8AM in 99% of the cases. I could then just set it and forget it and PGGB would process all the libraries over time without need for user interaction.

Basically 1 and 3 would solve the same problem for me, but with 1 I would get better utility rate for the computer as I could process music always when not listening. 3 however would already help a lot in timing the processing so that it doesn't intervene with music listening.

[LowOrbit]

I think there are some good user enhancement ideas there.

 

But can we stop it with the good suggestions for now and just get PGGBDSD out the door???  🤣

[HeeBroG]

On 4/13/2024 at 5:56 PM, sigmadelta said:

 

Can they confirm please DSD128 over DoP with SRC-DX?

 

With their Chord Dave

 

DSD128 does indeed work with SRC-DX/DAVE. I can't try DSD256 as I don't have any files atm because I have converted them to PCM 705/768 previously with PGGB. I'll get the native files from my office later this week to confirm that it does not work.

 

[Zaphod Beeblebrox]

5 hours ago, rayon said:

I also have quality-of-life thoughts related to stopping and starting the processing:

1. "Immediate pause/stop". Currently when clicking pause, it still spends a lot of time processing the current file, sometimes even 2h. I basically have to plan my listening sessions ahead of time as listening is stopping occasionally when PGGB is working in the background. In my dream state I could just click on Pause and PGGB would stop the processing immediately (releasing the computer for other use in 1 minute or so), release RAM and write cache into some temporary file.
2. Save before pause/stop (if the above is not feasible): When using Pause -functionality, it pauses right before it safes the file it just processed. That last step of just saving the file takes usually like 10 seconds or so and compared to time the processing takes, it's basically nothing. Especially in the beginning I ended up situations in which I paused it, then stopped and then realized that even if it finished, it actually never saved the result and I had to do that heavy calculation again. I would suggest that when pausing during the processing, it also saves the file and then pauses. Same for stop. I'd guess that this shouldn't be too many lines of code, but would make it behave much more logically.
3. Time window when it's allowed to start new work. I'd guess that "immediate pause" is hard to implement as it's relying on mathlab under the hood, but if user could give PGGB time window for when it can start new work, I could put there something like 24-06 as I basically newer listen to music during the night and even if it started to process something right before 6AM, it would be done before 8AM in 99% of the cases. I could then just set it and forget it and PGGB would process all the libraries over time without need for user interaction.

Basically 1 and 3 would solve the same problem for me, but with 1 I would get better utility rate for the computer as I could process music always when not listening. 3 however would already help a lot in timing the processing so that it doesn't intervene with music listening.

Again, good ideas. As you alluded to, some of these are due to communicating between Matlab process and PGGB library.

 

Your 2. Makes sense, no point in throwing way when the processing was just done on a file. I have made this change, and it is simple to do.

 

Your 1. For the new release, I have decided to remove 'pause' as it is not useful the way it is implemented now, I will add pause again once I built the mechanism to pause relatively quickly (this will be a later update, not on the first release)

 

Stop will continue to work like it does now, it will stop once the current track is done. But once I improve how pause works, stop will also work relatively quick (this will be a later update, not on the first release)

 

Your 3. I think if pause/start works as intended, there is very little need to do 3 and add additional complexity.

 

[Schafheide]

I have just gargle blasted my first Mahler symphony @ 20bit/16fS/256.  The two longest tracks have been split into 2 parts, as expected.  But in playback (HQP) they remain separate, instead of playing as one long movement.  Why is it so?  How do I fix this?

[Zaphod Beeblebrox]

4 minutes ago, Schafheide said:

I have just gargle blasted my first Mahler symphony @ 20bit/16fS/256.  The two longest tracks have been split into 2 parts, as expected.  But in playback (HQP) they remain separate, instead of playing as one long movement.  Why is it so?  How do I fix this?

When you say, 'remain separate', do you mean there is an audible gap? if so, how long? Where are you playing it from, is it local storage or NAS?

[Schafheide]

As you were!  I jumped in without fully analysing the problem. It was due to minor track jumbling, probably caused by the very long track names.  I know that this can be rectified using a tag editor, but it is a nuisance.  Sorry to alarm you!

[Zaphod Beeblebrox]

4 minutes ago, Schafheide said:

As you were!  I jumped in without fully analysing the problem. It was due to minor track jumbling, probably caused by the very long track names.  I know that this can be rectified using a tag editor, but it is a nuisance.  Sorry to alarm you!

If you had processed the whole album, then PGGB should renumber the tracks even if they were split and they should play in order. The one-time renumbering will not work is if a single folder had multi-disc tracks, then the track numbers can repeat. Another way to avoid this completely will be to use wavpack compression. You can click on the cocktail image and it reveals the hidden menu. there you can just select FLAC/Wavpack compression. Some avoid using wavpack due to the additional processing needed to decode YMMV.

[Schafheide]

Thank you.  I am just about to try wavpack. Cheers.

[Schafheide]

Another thought.

My Mac both gargle blasts and serves the files on it's 1 integral SSD.

Whilst trying to clean up what I thought were unimportant files, I accidentally deleted the licence file (I had a backup).

Perhaps those files, that are unimportant, could put in an obvious folder during the processing and critical files be hidden from us inveterate fiddlers?

[Zaphod Beeblebrox]

8 minutes ago, Schafheide said:

Another thought.

My Mac both gargle blasts and serves the files on it's 1 integral SSD.

Whilst trying to clean up what I thought were unimportant files, I accidentally deleted the licence file (I had a backup).

Perhaps those files, that are unimportant, could put in an obvious folder during the processing and critical files be hidden from us inveterate fiddlers?

Please DM me which files that you are referring to. The most useful log files are in the output folder of the corresponding album folders.

[Schafheide]

Actually I may have caused the file confusion problem.

Before installing PGGB 256, I thought that it might be useful if I separated my existing album folders from the gargle blasted album folders. So I created a folder called PGGB. 

I then installed PGGB into it's default location (PGGB).

So I ended up with a folder containing both application and music files DUH!

My gargle blasted folders are now in Music (the default macOS folder for such things).

All is now sweet!

[rayon]

On 4/6/2024 at 4:22 PM, seeteeyou said:

For those of us who aren't familiar with the differences between regular DDR4 and Intel Optane Persistent Memory, here are some examples

 

https://glesys.se/blogg/benchmarking-intel-optane-dc-persistent-memory

https://www.hpcwire.com/2020/02/19/japans-aist-benchmarks-intel-optane-cites-benefit-for-hpc-and-ai/

https://infohub.delltechnologies.com/en-US/l/r740xd-servers-with-intel-optane-dc-persistent-memory-handle-more-than-with-nand-flash-nvme-or-sata-ssds-2/performance-results/

 

Used 512GB sticks are going for $219 a piece or best offer

 

https://www.ebay.com/itm/126343937974

 

Even cheaper ones were also available but not sure about the longevity since they wouldn't last as long as the "immortal" DDR4

 

https://www.ebay.com/itm/115939077343

 

128GB sticks are like 50 bucks or so a piece

 

https://www.ebay.com/itm/296134704782

https://www.ebay.com/itm/305353892919

 

---

 

Asus WS C621E Sage is it still worth it

https://forum.level1techs.com/t/asus-ws-c621e-sage-is-it-still-worth-it/208746/2

 

Xeon Platinum 8259CL

https://www.cpu-world.com/CPUs/Xeon/Intel-Xeon 8259CL.html

https://www.cpumodel.com/cpu/intel-xeon-platinum-8259cl-24-cores-lga3647-server/

 

Xeon Platinum 8272CL

https://www.cpu-world.com/CPUs/Xeon/Intel-Xeon 8272CL.html

https://www.cpumodel.com/cpu/intel-xeon-platinum-8272cl-26-cores-lga3647-server/

 

Xeon Platinum 8273CL

https://www.cpu-world.com/CPUs/Xeon/Intel-Xeon 8273CL.html

https://www.cpumodel.com/cpu/intel-xeon-platinum-8273cl-28-cores-lga3647-server/

 

Xeon Platinum 8275CL

https://www.cpu-world.com/CPUs/Xeon/Intel-Xeon 8275CL.html

https://www.cpumodel.com/cpu/intel-xeon-platinum-8275cl-24-cores-lga3647-server/

 

All of them are well over 20 cores while they're also able to support Optane DCPMM mentioned above, therefore PGGB should be able to reap the benefits then?

 

8272CL with 26 cores turned out to cost about $269

 

https://www.ebay.com/itm/144688611079

https://www.ebay.com/itm/175151808032

https://www.ebay.com/itm/385618513299

 

650 bucks (or best offer) for 8273CL with 28 cores

 

https://www.ebay.com/itm/255969531554

 

Even cheaper here

 

https://www.bargainhardware.co.uk/intel-xeon-platinum-8273cl-srf81-28-core-2-20ghz-lga3647-38-5mb-165w-cpu

 

---

 

Finally it's just a matter of picking the right motherboard.

 

ATX-sized ones would come with relatively weak VRMs and therefore maximum TDP for the CPU turned out to be 165W

 

https://www.provantage.com/supermicro-mbd-x11spm-f~7SUPM5QR.htm

https://www.acmemicro.com/Product/15828/Supermicro-X11SPM-F-Motherboard-Intel-Xeon-Scalable-Single-Socket-P-

https://www.dihuni.com/product/supermicro-mbd-x11spm-f-o-x11spm-f-server-motherboard-intel-chipset-socket-p-lga-3647/

 

We could also go for E-ATX with beefier VRMs (up to 205W TDP) as well as 12 DIMM slots, prices should be just under 700 bucks at the moment

 

https://www.ebay.com/itm/304932239836

https://www.ebay.com/itm/325924480668

https://www.ipcstore.com/supermicro-mbdx11spatfo-motherboard-intel-c621-lga-3647-socket-p-extended-atx-3010579

 

Other than that, check this out if we're interested in VRM mods since the kit would cost only 50 bucks

 

https://forums.servethehome.com/index.php?threads/vrm-modify-icc_max-to-run-high-current-tdp-oem-processors.38686/

 

Previous releases of Supermicro BIOS

 

https://drunkencat.net/misc/SupermicroBIOS.html

 

---

 

Intel Xeon Platinum 8273CL Complete Info Guide

https://forums.servethehome.com/index.php?threads/intel-xeon-platinum-8273cl-complete-info-guide.41479/

 

512 GB Intel Optane DDR4-2666 RDIMM PMEM NMA1XBD512GQS

https://forums.servethehome.com/index.php?threads/512-gb-intel-optane-ddr4-2666-rdimm-pmem-nma1xbd512gqs.40019/

 

Basically the caveat emptor here should be all about the longevity of used Optane DCPMM since they could have the potential to be REALLY worn out already

 

https://seitics.de/files/ipmctl/

https://forums.servethehome.com/index.php?threads/512-gb-intel-optane-ddr4-2666-rdimm-pmem-nma1xbd512gqs.40019/page-3#post-379460

https://forums.servethehome.com/index.php?threads/512-gb-intel-optane-ddr4-2666-rdimm-pmem-nma1xbd512gqs.40019/page-4#post-388714

 

---

 

While Optane DCPMM might not be as good as the regular DDR4, it's still a relatively affordable way to boost the performance while getting such an impressive amount of persistent memory.

 

Always remember to RTFM since we've gotta pay attention to the ratio between regular DDR4 and Optane DCPMM

 

https://lenovopress.lenovo.com/lp1167.pdf

https://lenovopress.lenovo.com/lp1192.pdf

Hmm, so I'd guess that those optane DIMMs are the ones to look for and just skip those that go to NVME? I only have two dimm slots (didn't foresee this use case coming), but I do have free NVME slots. I wonder if those NVME optanes also would give an order of magnitude speed boost over regular NVME or if it must be in DIMM slot.

 

[seeteeyou]

2 hours ago, rayon said:

I wonder if those NVME optanes also would give an order of magnitude speed boost over regular NVME or if it must be in DIMM slot.

 

In terms of caching purposes, we'll need at least 1TB for PCIe drives while the U.2 version of Optane 905P would still cost way too much at the moment even if they're already discontinued by Intel

 

https://www.ebay.com/itm/294399688963

https://www.newegg.com/intel-optane-ssd-905p-series-960gb/p/N82E16820167463

While Optane should be great in terms of both longevity and latency, the sequential write speed should be THE most important for caching purposes. We're gonna get a much better deal with any PCIe Gen 4 or even Gen 5 drives (i.e. the M.2 2280 ones) these days since it's just inevitable that the latest models would totally "smoke" any relatively old Optane drives.

 

---

 

As long as we've got free PCIe x16 slots that are connected to the CPU directly, read the manual of the motherboard and make sure that PCIe Bifurcation is actually supported. Then we could get one of these PCIe adapters in order to install 4 M.2 2280 NVMe drives

 

https://shop.asus.com/us/90mc0cy0-mvaay0-hyper-m-2-x16-gen5-card.html

https://www.asus.com/motherboards-components/motherboards/accessories/hyper-m-2-x16-gen5-card/

https://www.asus.com/motherboards-components/motherboards/accessories/hyper-m-2-x16-gen-4-card/

 

---

 

BTW, my post about Optane DCPMM should be more about taking advantage of those REALLY high core counts of (Cascade Lake-SP) 2nd Gen Xeon Scalable processors since Intel reserved specific SKUs for their cloud computing customers such as Amazon etc. We're gonna get a whopping 48 to 56 threads for cheap, depending on the number of cores available. OTOH, stuff like i9-14900K would provide only 32 threads to begin with and supposedly it's drawing over 400W during peak power

 

https://www.anandtech.com/show/21084/intel-core-i9-14900k-core-i7-14700k-and-core-i5-14600k-review-raptor-lake-refreshed/6

 

Total system power consumption

 

https://www.custompc.com/intel/core-i9-14900k-review

https://en.overclocking.com/review-intel-i9-14900k-i7-14700k-and-i5-14600k/7/

 

If something were consuming THAT much power to begin with, then I'd rather consider CPUs with a relatively high number of threads since PGGB should be able to reap the benefits accordingly.

 

(The cheapest Optane DCPMM would only work with specific models of 2nd Gen Xeon Scalable, though.)

[Schafheide]

Another suggestion.

In my present stage of establishing the best settings (re processing time) - I try some settings and decide they are not what I want, so I want to abandon the process, delete the partially processed folder and start over with new settings.  It would be more convenient if, when I pressed "Stop", processing immediately stopped.  Perhaps this is not possible?

[Zaphod Beeblebrox]

52 minutes ago, Schafheide said:

Another suggestion.

In my present stage of establishing the best settings (re processing time) - I try some settings and decide they are not what I want, so I want to abandon the process, delete the partially processed folder and start over with new settings.  It would be more convenient if, when I pressed "Stop", processing immediately stopped.  Perhaps this is not possible?

Correct, it is not possible to do that right now, but I will consider implementing it later.

[Zaphod Beeblebrox]

In preparation for the imminent release of PGGB Plus (DSD capable PGGB), I wanted to share some more information and answer any questions you may have:

• Why DSD now?
• Quick Start Guide to PGGB DSD
• DSD Modulator performance
  • By DSD Rates
  • By Modulator Order
• System, OS and memory requirements

• On Memory and Speed, why is PGGB so demanding?

• Sample PC/ Mac builds for PGGB

PS: While I developed PGGB DSD with existing users in mind, allowing existing PC/Macs to do DSD conversions, to do conversion of full libraries, a PC/Mac with 128GB of RAM and at least 16 core processor and at least 512GB of free space for paging with enough cooling to avoid thermal throttling will take you a long way. I am also actively working on improving speeds even further.

[Mista Lova Lova]

What a pain upsampling to DSD at this level of precision is, from the point of view of the resources required. But then it must really speak volumes about how good the final result is if a usually-impatient person like myself is willing to go to these lengths.

 

Exciting times.

[taipan254]

3 hours ago, Zaphod Beeblebrox said:

In preparation for the imminent release of PGGB Plus (DSD capable PGGB), I wanted to share some more information and answer any questions you may have:

• Why DSD now?
• Quick Start Guide to PGGB DSD
• DSD Modulator performance
  • By DSD Rates
  • By Modulator Order
• System, OS and memory requirements

• On Memory and Speed, why is PGGB so demanding?

• Sample PC/ Mac builds for PGGB

PS: While I developed PGGB DSD with existing users in mind, allowing existing PC/Macs to do DSD conversions, to do conversion of full libraries, a PC/Mac with 128GB of RAM and at least 16 core processor and at least 512GB of free space for paging with enough cooling to avoid thermal throttling will take you a long way. I am also actively working on improving speeds even further.

Hi ZB - thanks for all this material. 
 

How should we think about the tradeoffs between PGGB PCM and PGGB DSD for both the lowest possible rates (PGGB 64 PCM vs highest available dsd option for that license) and highest possible rates PGGB 256 PCM vs. DSD)? 
 

For example, it seems that at the high end PGGB 256 PCM has lower noise floor in audible range allowing for better small signal reconstruction and less time domain precision relative to PGGB DSD 1024 at 9th order modulator because of the higher sample rate of dsd?

 

Hopefully this question makes sense!