HQPlayer4 EC modulator tips and techniques

[asdf1000]

1 minute ago, k6davis said:

You just have install the driver. Being a Linux noob myself, I looked up how to do that. There are a few commands, one figures out what the latest driver is & downloads it (something about "ppa"). The next actually installs it. I don't remember the specifics, but they are easy to find.

 

No worries. It was the top Google result:

 

https://askubuntu.com/questions/18747/how-do-i-install-run-files

 

Overall looks easier than I expected.

 

Thanks for the tips!

[asdf1000]

One final question @k6davis (and I'm sure all this will help others in future).

 

If installing Ubuntu Server , how to see GPU usage via command line, to check that the GPU is actuall being used by HQP? And to see how much GPU is active.

 

I've always used top command but for CPU.

 

Is there a simple and recommended command?

 

[k6davis]

12 minutes ago, asdf1000 said:

If installing Ubuntu Server , how to see GPU usage via command line, to check that the GPU is actuall being used by HQP? And to see how much GPU is active.

 

I've always used top command but for CPU.

 

Is there a simple and recommended command?

 

This will do it.

 

Like I said, I'm a Linux noob, but I typed in the commands from here verbatim and it worked perfectly. 

 

The only thing they didn't explicitly state was that you should put "sudo" in front of the last command.

make install # You may need sufficient permission for that (root)

... will likely need to be...

sudo make install # You may need sufficient permission for that (root)

.. which you wouldn't know if you don't know anything about Linux, but it appears that you do @asdf1000.

 

It's a nice GPU monitoring app.

[sledwards]

Once you get the latest driver installed, you can type: sudo nvidia-smi to get GPU process info too.

[k6davis]

5 hours ago, sledwards said:

Once you get the latest driver installed, you can type: sudo nvidia-smi to get GPU process info too.

 

This is correct and is quick, easy and effective for verifying that you GPU/Cuda is working. It provides a single, static snapshot of the GPU usage stats, which can be all you need.

 

 

nvtop, the more involved solution I found (2 posts ago), gives you continuous and graphical monitoring in color, more like Windows Task Manager, or Linux "htop". 

 

[tboooe]

On my i7-6700 playing redbook, I noticed CPU utilization went down a bit when I unchecked the Multicore DSP option.  

[k6davis]

37 minutes ago, tboooe said:

On my i7-6700 playing redbook, I noticed CPU utilization went down a bit when I unchecked the Multicore DSP option.  

 

Yes. Multicore can behave in mysterious ways so it should be experimented with - especially when Cuda is also being used.

 

Also, try DSD5EC. It has a lower CPU load and I really like the sound of it.

[Miska]

11 hours ago, asdf1000 said:

2. Install latest driver from here:

https://www.geforce.com/drivers/results/149138

 

Is that all?

 

You only need latest GPU drivers, nothing else... So just this.

 

[Miska]

11 hours ago, asdf1000 said:

but "nano /usr/share/doc/hqplayerd/readme.txt.gz/readme.txt"  doesn't work for me.

 

Edit: ran "gzip -d readme.txt.gz"

 

Usually "less" knows how to display compressed text files. Otherwise you can pipe "gzip -d readme.txt.gz | less".

 

[Miska]

16 hours ago, shadowlight said:

My setup is Dell Server with quad Intel(R) Xeon(R) CPU E5-2643 v2 @ 3.50GHz six core server with 64GB of memory, running Ubuntu Bionic LTS with latest patches and latest version of embedded.  The result of top has cpu usage value around 1000 when using EC modulator and around 400 when using ASDM7.  I think I still have plenty of CPU cycles available, since 100% load across all CPU would be 2400 so I am currently under 50%.

@Miska

is there a way to optimize CPU load under embedded?

 

If you have 4 sockets of E5-2643 (quad core), that means total of 16 cores. So 1600 would be full load on all cores, not including threads.

 

However, if some of the cores reach close to 100% load individually, you are close to maximum and get capped by clock frequency of a single core. For EC modulators I would say you need at least four cores running minimum of 4 GHz, plus what ever is needed to run the filters (if not offloaded to GPU).

 

[Xoverman]

@miska 

Great filters. 

Do EC modulators use integer64 or 64Bit floating point? 

I can run EC filters to DSD256 on 4core CPU @4Ghz, do you think it would be possible to run DSD512 on 8 or 12core CPU @4GHz. Or doesn't it scale that whel? 

[shadowlight]

3 hours ago, Miska said:

 

If you have 4 sockets of E5-2643 (quad core), that means total of 16 cores. So 1600 would be full load on all cores, not including threads.

 

However, if some of the cores reach close to 100% load individually, you are close to maximum and get capped by clock frequency of a single core. For EC modulators I would say you need at least four cores running minimum of 4 GHz, plus what ever is needed to run the filters (if not offloaded to GPU).

 

 

The server in question has 6 cores per cpu - E5-2643 v2.  I will look into seeing if there is something that I can do at the Bios level.  Can I do anything similar @ OS / Embedded level.  Multicore is currently set @ auto.

 

From hqplayerd log.

 

2019/08/11 23:09:52 Parallel threads: 24
2019/08/11 23:09:52 Nested parallelism: 4
2019/08/11 23:09:52 Parallel pipelines: 4

 

[agillis]

On 8/8/2019 at 6:00 AM, asdf1000 said:

 

Maybe @agillis @vortecjr can advise if they can build a fanless i9-9900K and price, since a few may be interested here?

 

 

We do sell an i9 based system already.

 

https://www.smallgreencomputer.com/collections/audio-server/products/sonictransporter-i9-for-roon-dsp

 

 

[asdf1000]

14 minutes ago, agillis said:

 

We do sell an i9 based system already.

 

https://www.smallgreencomputer.com/collections/audio-server/products/sonictransporter-i9-for-roon-dsp

 

 

 

It’s not the i9-9900K (95W TDP) though, right?

 

The ‘K’ is what people have working, PCM192k to DSD256 with ext2 and ASDM7EC.

 

All discussed in this thread.

 

There is a 110W TDP Streacom case, DB4

[asdf1000]

On 8/8/2019 at 7:18 AM, Miska said:

My hardware documented here:

 

It can do DSD256 with the EC modulators. Multicore set to "auto".

 

Embedded on Ubuntu Server 18.04.

 

Can yours do DXD (PCM353kHz) to DSD256 with ASDM7EC, ext2 filter?

 

[asdf1000]

Switching gears from Intel for a moment

 

Has anyone got an AMD Ryzen 7 3800X (base freq = 3.9 GHz) working, PCM192 to DSD256 with ASDM7EC + ext2.

 

On HQP Embedded.

 

Either with or without CUDA offloading.

[tedwoods]

21 hours ago, tboooe said:

On my i7-6700 playing redbook, I noticed CPU utilization went down a bit when I unchecked the Multicore DSP option.  

I have to have CUDA offload checked for it to work properly, but multi-core greyed and not checked for the CPU to work below 30%.

I don't know if that's how it's supposed to behave though...

[Miska]

3 hours ago, tedwoods said:

I have to have CUDA offload checked for it to work properly, but multi-core greyed and not checked for the CPU to work below 30%.

I don't know if that's how it's supposed to behave though...

 

When multicore is grayed, HQPlayer calculates if there are suitable number of real CPU cores available for the case and decides whether to parallelize certain operations or not (knowing how many threads there would be vs how many cores are available). When multicore is checked, all parallelization is blindly turned on regardless how many CPU cores there are vs the needed workload.

 

Calculation is always an estimate, so is some cases, forcing everything in parallel may yield better results when things are really on the edge, but this is really up to experimentation in each particular case and configuration. In most cases, grayed (auto) option should yield in fairly optimal results.

 

[k6davis]

20 minutes ago, Miska said:

When multicore is grayed, HQPlayer calculates if there are suitable number of real CPU cores available for the case and decides whether to parallelize certain operations or not (knowing how many threads there would be vs how many cores are available). When multicore is checked, all parallelization is blindly turned on regardless how many CPU cores there are vs the needed workload.

 

Calculation is always an estimate, so is some cases, forcing everything in parallel may yield better results when things are really on the edge, but this is really up to experimentation in each particular case and configuration. In most cases, grayed (auto) option should yield in fairly optimal results.

 

The forthcoming AMD Ryzen 9 3950X may present an opportunity for the best HQP performance yet, with 16 cores and 32 threads and the largest ever cache. But all of those cores run slightly slower than the cores of my i7-9700K. 

 

It will be very interesting to see if it's capable of anything that the i7-9700K is not regarding HQP EC. 

 

[Le Concombre Masqué]

30 minutes ago, Miska said:

 

When multicore is grayed, HQPlayer calculates if there are suitable number of real CPU cores available for the case and decides whether to parallelize certain operations or not (knowing how many threads there would be vs how many cores are available). When multicore is checked, all parallelization is blindly turned on regardless how many CPU cores there are vs the needed workload.

 

Calculation is always an estimate, so is some cases, forcing everything in parallel may yield better results when things are really on the edge, but this is really up to experimentation in each particular case and configuration. In most cases, grayed (auto) option should yield in fairly optimal results.

 

guess it's chance rather than AI inside but for the first time I have been able to listen to, in a row, several DSD 64 tracks turned 128+Convolution with DSD5EC

[Miska]

20 hours ago, shadowlight said:

The server in question has 6 cores per cpu - E5-2643 v2.  I will look into seeing if there is something that I can do at the Bios level.  Can I do anything similar @ OS / Embedded level.  Multicore is currently set @ auto.

 

From hqplayerd log.

 

2019/08/11 23:09:52 Parallel threads: 24
2019/08/11 23:09:52 Nested parallelism: 4
2019/08/11 23:09:52 Parallel pipelines: 4

 

Ahh, yes, v2 is 6-core. What does the log say about number of CPU cores? Is it correct for total?

 

For filters, this is a very good setup with high memory bandwidth for each CPU. I could optimize such configurations further and hopefully will do so in near future.

 

Challenge with modulators is that there's a limit how much the work can be parallelized, rest relies on raw core performance. Multi-socket helps on that, since each socket can have fewer cores at higher clock speeds.

 

 I will continue trying to optimize the new modulators to run on wider variety of hardware.

 

[shadowlight]

13 minutes ago, Miska said:

 

Ahh, yes, v2 is 6-core. What does the log say about number of CPU cores? Is it correct for total?

 

For filters, this is a very good setup with high memory bandwidth for each CPU. I could optimize such configurations further and hopefully will do so in near future.

 

Challenge with modulators is that there's a limit how much the work can be parallelized, rest relies on raw core performance. Multi-socket helps on that, since each socket can have fewer cores at higher clock speeds.

 

 I will continue trying to optimize the new modulators to run on wider variety of hardware.

 

I will take a look at it tonight.  Is there a quick way to identify how many cores HQPd is seeing from the logs?

[Miska]

15 minutes ago, k6davis said:

The forthcoming AMD Ryzen 9 3950X may present an opportunity for the best HQP performance yet, with 16 cores and 32 threads and the largest ever cache. But all of those cores run slightly slower than the cores of my i7-9700K. 

 

It will be very interesting to see if it's capable of anything that the i7-9700K is not regarding HQP EC. 

 

I will certainly try the new AMD chip when it becomes available here. Large cache should notably help with filters.

 

[Miska]

4 minutes ago, shadowlight said:

I will take a look at it tonight.  Is there a quick way to identify how many cores HQPd is seeing from the logs?

 

Yes, quite early on you have this kind of report:

  2019/07/24 23:48:23 Number of processor cores: 4
  2019/07/24 23:48:23 DSP thread pools disabled
  2019/07/24 23:48:23 DSP pipelines disabled
  2019/07/24 23:48:23 Pipelined DSP engine enabled

 

[Miska]

On 8/11/2019 at 11:09 PM, Xoverman said:

I can run EC filters to DSD256 on 4core CPU @4Ghz, do you think it would be possible to run DSD512 on 8 or 12core CPU @4GHz. Or doesn't it scale that whel? 

 

You'll need to scale clock speed instead of cores. That's what makes it tricky...