AudioLinux and NUC Troubleshooting and Tuning

[ray-dude]

 

On the mother thread, @Narcissus asked about Ramboot:

 

But don't you have to physically punch the "y/n" on the keyboard for entering ramroot? And then click on "Roon bridge", "HQPe" or Squeezelite whatever you're pairing up your server with?

 

It does "auto yes"  For convenience when running headless, I made the following change:

 

Edit /usr/bin/ramroot and change
LOAD_TIMEOUT_DEFAULT=15

 

to 

 

LOAD_TIMEOUT_DEFAULT=3

 

That changes the wait time at the "Y/N" prompt from 15 seconds to 3 seconds

 

 

[ray-dude]

@austinpop Thank you for the additional impressions. May I ask where you ended up with your end point SqueezeLite parameter settings?  (your /etc/squeezelite.conf )

 

I've just stood up my NUC7CJYH as a Roon Bridge end point, and want to experiment with the Roon Server/SqueezeLite end point configuration that you developed (alas, I confess that the myriad of SqueezeLite parameters are all still pig latin to me)

 

[ray-dude]

1 minute ago, mourip said:

When in headless mode and I run alconf and chose option #1 "Save system (If in ram mode)" does this mean that if I run that script it will save my settings if my USB drive is still in place?

 

So what happens if I am in ram mode, pull the USB drive and then make a change? Is that something that basically only affects that session and is gone when I reboot?

 

I assume that if I am not in ram mode and I make a change it will always be saved to my USB drive for the next boot?

 

I was playing with this last night.  the alconf option 1 invokes a script to copy things back to the USB.  If you are in RAM mode and the USB is removed, the script basically errors out.  If the USB is inserted and you're in RAM mode, the changed files will be copied back to the USB while you continue to run out of RAM.

 

If you're not in RAM mode, the changes you're making are direct to the USB and available for the next reboot

 

 

[ray-dude]

7 hours ago, austinpop said:

 

Yes, this is the reality of NUC-world. It's a bit wild west. 

 

As regards squeezelite, the main parameter for SQ is the -b one for preallocated buffers. I was lucky that my first experiment with AL was on a Zenith SE that already used squeezelite as its internal player. I was able to duplicate the settings in the existing conf file.

 

Other than that it's trial and error and forums! I can try and help out, but no guarantees. First, get me a few key things:

• install squeezelite (not R2) from the AL guide
• amount of RAM on your NUC?
• Run squeezelite -l and tell me your output
• Do you care about DSD? Does your DAC require DoP?

We'll go from there.

 

Thank you Rajeev

 

I just got my NUC7CJYH w/ 8GB up and running headless in RAM root (thank you and everyone here).  Runs great as a Roon Bridge, connected via USB to my Chord Blu2 (then Chord DAVE DAC).  Upstream I have a vanilla Mac Mini (OSX) running my Roon Core, not yet optimized.  With this setup, I need the end point to just pass things bit perfect to the BluDAVE and let it do its (considerable) magic.

 

The tunability of the Squeezelite end point (vs Roon Bridge) seems like a great opportunity to dig in and figure out what is driving the SQ boosts we're all hearing.

 

 

 

Quote

 

[audiolinux@audiolinux ~]$ squeezelite -l

Output devices:

  null                           - Discard all samples (playback) or generate zero samples (capture)

  default:CARD=Blu2              - Blu2, USB Audio - Default Audio Device

  sysdefault:CARD=Blu2           - Blu2, USB Audio - Default Audio Device

  front:CARD=Blu2,DEV=0          - Blu2, USB Audio - Front speakers

  surround21:CARD=Blu2,DEV=0     - Blu2, USB Audio - 2.1 Surround output to Front and Subwoofer speakers

  surround40:CARD=Blu2,DEV=0     - Blu2, USB Audio - 4.0 Surround output to Front and Rear speakers

  surround41:CARD=Blu2,DEV=0     - Blu2, USB Audio - 4.1 Surround output to Front, Rear and Subwoofer speakers

  surround50:CARD=Blu2,DEV=0     - Blu2, USB Audio - 5.0 Surround output to Front, Center and Rear speakers

  surround51:CARD=Blu2,DEV=0     - Blu2, USB Audio - 5.1 Surround output to Front, Center, Rear and Subwoofer speakers

  surround71:CARD=Blu2,DEV=0     - Blu2, USB Audio - 7.1 Surround output to Front, Center, Side, Rear and Woofer speakers

  iec958:CARD=Blu2,DEV=0         - Blu2, USB Audio - IEC958 (S/PDIF) Digital Audio Output

 

 

My /etc/conf.d/squeezelite settings are:

 

parameters="-o front:CARD=Blu2,DEV=0 -b 2097152:2097152"

 

(piggybacking on your findings that maximizing buffers makes things better, although I'm wondering if weighting more towards the input or output would move the SQ needle...TBD)

 

However, I was also looking at the real time priority, parameters for connecting to the DAC, and even codec settings as potential SQ levers (TBD how the stew of latency and compute and OS priorities all come together when you expand the test matrix)

 

I hoping to dive in this weekend and start doing some investigations.  If you (or others) have run into dead ends or opportunity areas with Squeezelite as the end point, it be great to build on that!

 

 

 

[ray-dude]

A quick WIP update on some listening experiments this weekend.

 

Background:

 

I recently replaced a microRendu 1.4 with Sbooster LPS with a ram booted NUC7CJYH w/ single 8GB stick running AudioLinux (now) 0.6 (stock PS while I wait on a new LPS).  

 

I have Roon Core on a stock Mac Mini (OSX) connected by vanilla ethernet through vanilla switches to the NUC, and NUC is connected by vanilla USB to a Chord Blu2+Chord DAVE stack.  I was surprised and intrigued by the finding that squeezelite (with large buffers) was a better end point than Roon Bridge, and I've been fiddling with squeezelite settings to try and suss out what's going on.

 

Running squeezelite with "-b 2097152:2097152" (2GB application buffer on each of stream input and DAC output) I immediately heard a significant increase in detail and depth, confirming other's experiences in my setup (surprising lift over Roon Bridge and the squeezelite default).  Per Rajeev's observations, the larger the buffer, the larger the lift (again intriguing...these music files are WELL below 2GB in size)

 

Some baseline file transfer benchmarks:

 

Using sftp, I moved some large files between my Mac (SSD) and NUC (ram boot) and saw transfer rate of 80-90 MB/sec (as expected).  On the NUC itself, duplicating the same files I saw ~570 MB/sec (consistent with the ~600 MB/sec I see when running memorytest on the NUC).

 

As expected, gigabit ethernet is slower than memory (you'd be surprised how often I have to remind my work colleagues about that

 

Since memory is limited on the NUC, and more memory allocated to buffer is improving SQ, would allocating more to the upstream (ethernet) side of the bridge move the SQ needle?  At first blush, answer seems to be "Yes", but (intriguingly) so does allocating more to the downstream (DAC) side, but to a lesser degree.

Below is my /etc/conf.d/squeezelite file for the buffer settings I tested.

 

# Default

#parameters="-o front:CARD=Blu2,DEV=0"

# Front

parameters="-o front:CARD=Blu2,DEV=0 -b 4097152:97152"

# Back

#parameters="-o front:CARD=Blu2,DEV=0 -b 97152:4097152"

# Balanced

#parameters="-o front:CARD=Blu2,DEV=0 -b 2097152:2097152"

# Original

#parameters="-o iec958:CARD=Audio,DEV=0"

 

 

Of these, both "Front" and "Back" were better than "Balanced" (detail, soundstage, depth, etc), but with a small edge going to the "Front" config.  I have not yet done any systematic testing with less extreme mixes. 

 

It seems like larger buffer matters, no matter what side of the bridge it is (wild)  My test files are on the order of 10's of MB (<<1 second to transfer complete file from the server to the NUC), so how the heck can 2GB vs 4GB buffer (regardless of side) make a meaningful difference?   What is the extra buffer doing to improve the decoupling/isolation?

 

My WAG is something wacky is happening on the Roon Core side and how it interacts with the squeezelite end point (which would be consistent with the weird behavior I have occasionally seen when using Roon to seek within tracks or skipping between tracks).  It would be interesting to see if a different core server (eg, LMS) has the same behavior as buffer sizes increase.

 

Next, time to play with the ALSA parameters in squeezelite (-a parameters).  I suspect those results will be much more specific to my Chord BluDAVE DAC, but (alas) so far my intuition is batting pretty close to 0.000 when it comes to these NUCs.  Brave new world indeed.

 

 

 

 

[ray-dude]

3 hours ago, austinpop said:

Awesome work!

 

I did some analysis last night as well, and will post my findings once I get to a keyboard later this afternoon.

 

Stay tuned.

 

Looking forward to your report Rajiv!  

 

I was able to spend some time with the squeezelite ALSA parameters before fatigue set in.  From the help page:

 

Quote

-a <b>:<p>:<f>:<m> Specify ALSA params to open output device, b = buffer time in ms or size in bytes, p = period count or size in bytes, f sample format (16|24|24_3|32), m = use mmap (0|1)

 

It is frustratingly difficult to find out what these various parameters mean.  From @bibo01 we've learned that the developer of squeezelite r2 uses "-a 499:3::0".  The default is "-a 40:4".  Others have posted experiences with "-a 499:3"

 

From my hair pulling (aka reading) sessions, I suspect the second parameter is very DAC dependent, but that is a pure guess.  In my experimentation with my BluDAVE stack, I found 4 to be the best, but it was very limited experimentation.

 

I spent most of my morning trying different ALSA circular buffer time values.  Note that when the buffer time value is 500 or greater, it is interpreted as bytes, and when it is below 500, it is interpreted as milliseconds.  This controls the size of the circular ALSA buffer that sits between squeezelite and your DAC (ALSA is the OS-level sound drivers in linux).  The period value reflects how the various channels are interleaved in the buffer, and the format value reflects how many bytes to reserve for each value.  I got lost in the ALSA docs and source code any deeper than that, so please forgive me if I have any of this wrong.

 

My strategy was to peg the period at 4, and leave the format and mmap values as default (in my case "any" and 1)

 

For those that want to play at home. at the command line, I was running squeezelite manually to make it easier to toggle various parameters:

 

/usr/bin/squeezelite -o front:CARD=Blu2,DEV=0 -b 4097152:97152 -d all=debug -a <various>:4 -c pcm -x

 

Execute the command, give a listen, control-c to kill it, modify the parameters, rinse and repeat

 

Breaking this down, -o points squeezelite at my DAC, the -b reflects stream and output buffer memory that gets reserved on start up (see my previous post above), -d sets out the log output level so I can follow the test in the terminal window (which is very informative!), -c restricts squeezebox to only use the PCM codec, and -x says "don't as the server to downsample anything".  

 

-x and -c should be no ops, since Roon Core is streaming PCM to the NUC, but I thought it wouldn't do any harm to disable features in squeezelite and keep things simple.

 

For these experiments. -a parameter is the one of interest to me, as it manages the output circular buffer between squeezelite and the OS and the DAC.

 

Using primarily Redbook content (my Blu2 does all my upsampling) that I am very familiar with, I tried <various> ALSA circular buffer sizes from 1000 bytes up to 1,000,000,000.  Since I'm just trying to figure out what all these parameters do (and if they matter), I used a limited number of audition tracks.  Fine tuning will come later.

 

The bottom line is I heard a notable uptick in SQ as I increased the circular buffer size, but things got progressively more flakey with Roon Core and latency went way up as the buffer size got to the top end.

 

I found my sweet spot to be between ~50-100MB (very sweet spot actually...delightful!):

 

/usr/bin/squeezelite -o front:CARD=Blu2,DEV=0 -b 4097152:97152 -d all=debug -a 100000000:4 -c pcm -x

 

To triangulate and spot check, a ~4:00 Redbook (44/16) ALAC file is ~27MB, ballooning to ~43MB when decoded to PCM.  My hypothesis is that if you can fit the entire PCM content in the ALSA circular buffer, you're as buffered as you're going to get.  Set the USB for your DAC to the highest IRQ RT priority and have squeezelite a close second in real time priority, and I'm not sure you can do any better (outside of continued OS tuning to get more and more real time)

 

So how do high res files (24 bit and higher sampling rate, etc.) and DSD files impact this?  Is this setting over optimized for Redbook content?  My ears and brain are too tired to check today  

 

My next step is to experiment with high res content (does 24 vs 16 bit changes the sweet spot?), DSD content (does it work ok?), and longer content (do I need a bigger circular buffer?).

 

The big next step is to see if an extra large ALSA circular buffer enables reallocating precious memory from the squeezelite buffer (the -b parameter above....currently heavily weighted on the streaming (ethernet) side) to the ALSA buffer between squeezelite and the DAC.  

 

In other words, if you're willing to live with the latency, does having a maximum ALSA circular buffer (large enough to fully buffer all your PCM content) make the upstream buffering (or running Roon core on the same NUC) less relevant to SQ?  I'm hoping the answer is yes!

 

 

 

 

 

 

[ray-dude]

Fascinating to look at the network traffic while things are running!  Really shines a light on what Roon core is doing (the most opaque part of our chains right now).  When I recover a bit, I'll repeat my ALSA circular buffer experiments while watching network traffic.  

 

I am very hopeful that maximizing the buffer at the OS level (ALSA) and as far away as possible from Roon Core will give us the biggest lift, and the most direct immunity from any transient behaviors from the player, etc. (my spider sense is that the -b buffers are giving us an important but indirect benefit, but the proof will be in the pudding!)

[ray-dude]

Thank you @mansr for the correction/redirect, and thank you @vortecjr (and Google translate) for the link to the blog post (wow, that clears up a lot of things!)   Time to brush up on my Italian and revisit my notes from today.  Thanks again!

[ray-dude]

I'm running SqueezeLite R2 1.8.4, but only because that was the first option on the AudioLinux install guide.  I haven't compared sound quality, but from the docs it looks like the functionality is aligned between R2 and the community version, but with different flags to enable.

 

squeezelite-R2

Lightweight headless squeezebox client emulator with 'native' DSD support (thanks to Daphile).

This is a modified version of squeezelite by Adrian Smith (Triode). At the moment of writing (October 2015), original code is here: https://code.google.com/p/squeezelite/, master branch here is a clone.

This version was originally meant to inspect pcm header to detect the real samplerate, depth and endianess, in order to override the wrong information coming from the server when transcoding or upsampling.

October, 4 2015 R2 features has been incorporated in Daphile, March, 10 2016 in Audiolinux.

Since March, 15 2016 it's included in the squeezebox community version of squeezelite, mantained by Ralph Irving.

Download page: https://github.com/marcoc1712/squeezelite-R2/releases

[ray-dude]

11 hours ago, BigAlMc said:

@ray-dude & @austinpop,

 

Awesome work guys and great discovery.

 

At the very real risk of pushing my luck it would be great if we could have a simple 'how to' guide for the technically incompetent (like me) showing how to set buffer size for:

 

Squeezelite endpoint

Roon bridge endpoint

 

And ideally in both the 4gb and 8gb flavours that are likely most prevalent.

 

The obvious question is whether the 4gb flavour has sufficient ram spare to make a difference in SQ via this buffer setting or whether 8gb is worth the investment?

 

Either way this is terrific stuff and an excellent area of exploration. Even if I need to read it twice in order to understand half of it! ?

 

Cheers,

Alan

 

 

Alan, I have 8GB, so I've not tried to cram things into 4GB.  With the SQ improvement we've heard with crazy large buffer sizes (still unclear why 2GB buffer is better than 20MB buffer for small audio files), I suspect you'll want to consider 8GB

 

Couple hygiene items above the usual AudioLinux install stuff (make sure to do this will ramboot off so you have all the changes on your USB stick)  I'm learning that one can't edit posts here, so if there are typos or incorrect information, my apologies in advance that I'm unable to correct.

 

* Set IRQ priority for the DAC USB to the highest: https://www.audio-linux.com/html/realtime.html

 

Run: rtstatus

This will show you the real time priorities for various applications and IRQs (HW interfaces)

 

Run: rtcards

This will show you your audio cards.  In my case, my DAC is card0, and sits on USB1 IRQ=123    IR-PCI-MSI 344064-edge xhci_hcd

 

Edit: /etc/rtirq.conf

 

Change

RTIRQ_NAME_LIST="usb"    <--- All the USBs have high priority
RTIRQ_PRIO_HIGH=90

 

to

RTIRQ_NAME_LIST="xhci_hcd"    <---- Only the DAC interface has high priority
RTIRQ_PRIO_HIGH=95

 

 

If you like, you can also make your ethernet a high priority by having it be:

RTIRQ_NAME_LIST="xhci_hcd eno1"

(order matters)

 

This may be helpful if you run Roon Bridge (which streams data continuously).  I am using SqueezeLite (more on that later), which buffers the data up front, so I do not have ethernet as a priority (I don't want random ethernet stuff impacting the system during playback)

 

 

* Set app priority for squeezelite and RoonBridge to be second highest

I have squeezelite as the highest priority application.  The order in the list reflects priority order if you have multiple apps running.  In practice, you'll only have one running, so not something I worried about.  I set the max priority for applications to be a hair less than the max priority for the IRQ to the DAC.

 

Edit:  /etc/rtapp/rtapp.conf

APPLICATIONS="squeezelite RoonBridge jackd mpd hqplayer hqplayerd RoonAppliance mediacenter24 networkaudiod deadbeef a2jmidid ardour-5.12.0 rosegarden audacity"

MAX_PRIORITY="93"

 

 

At this point, if you're running Roon Bridge, you should be able to reboot and confirm that everything is still working fine.

 

 

Now to the buffer stuff.

 

Alas, Roon seems to be a black box when it comes to buffers.  Per Rajiv's findings, it streams data from core to the end point continuously, and I'm not aware of a way to impact that.  Given how Roon works (multi-zone, responsive UI, etc.) that is unlikely to change.  All that behavior is made better (from a user experience stand point) by having minimal buffers.

 

If you want to play with buffers and try to isolate ethernet activity from streaming to the DAC, squeezelite is the end point of choice right now.  Alas, that comes with some "endearing quirks" (skipping, issues when seeking within a track, latency, etc), Whether the delta between convenience and SQ is something you can try and see if it is worthwhile for you.  For me, I'm leaving the system on Roon Bridge (for civilian use), except when I'm playing with maximizing SQ.  Squeezelite is a bit too quirky right now to impose on people that don't care about SQ.

 

First step is to install squeezelite (if you have AL headless...it is already installed on the full version).  From the AL site: https://www.audio-linux.com/html/user_guide1.html (copy/pasted here)

 

Quote

 

Software Install – Squeezelite-R2;

Commentary:
The install process asks you some questions:
It will ask you if you wish to edit some files = n, on all occassions.
When asked whether to continue building = y
If asked for a password = audiolinux or audiolinux0
Proceed with installation = y
When completed successfully it will report: No database errors have been found!

Steps:
> Type: yaourt -Sy
This will update the system
> Type: yaourt -S squeezelite-r2-git
This will load the squeezelite software
> Type: su
Enter the root password
> Type: squeezelite -l
Identify the USB output you are going to use
> Type: vi /etc/conf.d/squeezelite
Update squeezelite to use that output
> Type: systemctl daemon-reload
> Type: systemctl start squeezelite@audiolinux
> Type: systemctl status squeezelite@audiolinux
Should be shown as running
> Type: systemctl enable squeezelite@audiolinux
It will now start automatically at boot.

 

 

 

 

If you're tweaking squeezelite parameters to experiment in your system, I found it is easier to do it from the command line.  You start squeezelite, test it, hit control c to kill it, modify the parameters, start it again, rinse and repeat.

 

To do this, go to alconf and select "Stop and disable all running audio services"

 

From the command line, after some more experimentation, here is my current squeezelite config of choice in my system:

 

/usr/bin/squeezelite -o front:CARD=Blu2,DEV=0 -b 4097152:97152 -d all=info -a 1000000:4 -c pcm -x -p 93

 

For you, you may need to change some of the parameters.  

 

The -o parameter is the output device.  You get this info for your DAC by running: squeezelite -l

 

the -b parameter is the buffer memory that squeezelite preallocates for input from the network (first number) and output to the day (second number).  The unexplained finding is that bigger buffers are better.  I have it set so that the bulk of the buffer is allocated to the ethernet side.  The thinking here is that since we're hearing bigger buffers are better, and having an asymmetric buffer allows us to allocate more buffer space to the ethernet side.  Why 4GB vs 2GB matters for 20MB music files is a crazy making question, but I'm blaming Roon Core because they're not here to defend themselves

 

The -d parameter just sets the debug output level so you can see what squeezelite is doing as you experiment.  

 

The -a parameter manages the buffer between squeezelite and the OS (and eventually to the DAC).  This is still a mystery to me, but above are my current settings of choice (1MB allocated to the buffer, with a periodicity of 4).  I'm still working on my Italian to suss out what all these mean, and there is more learning/experimentation to do here.

 

-c says squeezelite only accepts PCM, and -x says no downsampling.  Shouldn't effect SQ, I just have them for hygiene.

 

-p sets the real time priority for the squeezelite output thread.  Since I set this 93, and I have the IRQ real time priority set to 95.  From rtstatus, everything else is 50 or below.

 

 

Once you have a config that you're happy with, you need to edit and add it to the following file: /etc/conf.d/squeezelite

 

parameters="-o front:CARD=Blu2,DEV=0 -b 4097152:97152 -a 1000000:4 -c pcm -x -p 93"

 

If you go to alconf and select "START and enable Squeezelite" next time you reboot, squeezelite will be started with these parameters

 

 

 

Aside from the tactics, here is my high level strategy/thinking for the recipe (all hypothesis driven, we haven't figured this stuff out yet).  I would love to hear criticism/correction/redirection on this from others.

 

Goal:

Get an end point as focused as possible on moving data from ethernet to a USB DAC, with as little variability and as much predictability as possible (aka, as close to precision real time data streaming to DAC as possible)
    - Maximize processing/memory allocated to bridging bits from server to end point, to preload data as quickly as possible
    - Have an end point doing as little as possible when streaming data to DAC (everything turned off, front load network access, etc)

    - Highest priority given to end point talking to DAC (DAC USB IRQ, squeezelite thread, etc)

    - Use a stripped down semi-real time OS (AudioLinux) doing as little as possible
    - Minimize power use (RF, etc.) by shutting everything unneeded off, running in memory, etc.

    - Use the highest power silicon on a chip system with reasonable power use you can find, and manage the trade off between lower latency (good) and higher power (bad) 

 

(as an aside, I'm OK with absolute latency, I just don't want the latency to vary...my working theory is that low latency systems are a proxy for having less variability, because they have faster CPUs, more cache, better components, etc, and less contention and more headroom at any given time)

Approach:

 

* NUC with high quality DC inputs, so you have all the goodies combined into single Silicon on a Chip implementation (assertion:

there is less variability with SoC)
* AudioLinux running headless, tuned for real time, OS running in RAM (max speed, lowest latency)

* End point software that allows tuning and optimization and buffering (squeezelite)

* Maximum buffers between server and end point (-b parameter), to preload data to memory over the network as quickly as possible

    - Isolate data streaming to DAC streaming from rest of chain as much possible, for as much of the music playing time as possible

* Maximum buffers and priority between end point and DAC (-a parameter)

    - Feed DAC the steadiest/most stable/least variable data stream as possible during music playback

* Have as little as possible other things happening during music playback

    - minimal network access, no storage access, unused features disabled, lower priority for all other things running in OS, etc
 

 

All the above is hypothesis, but a snap shot of my current thinking.  Many here have been working and experimenting and thinking about this stuff much longer than I have, so I would give their experiences and gut intuition much more credence (I'm still working my way up the learning curve with this kit).

 

 

[ray-dude]

27 minutes ago, Dutch said:

Gave my ethernet interface high RT priority as well as my USB port to which my DDC connects:

 

/etc/rtirq.conf

---
RTIRQ_NAME_LIST="xhci_hcd eno1"

# Highest priority.
#RTIRQ_PRIO_HIGH=90
RTIRQ_PRIO_HIGH=95

 

Looking forward to hearing how it sounds @Dutch

 

I went back and forth on whether the ethernet should have high priority.  

 

Between the the OG reports of improved SQ by network bridging (which I interpreted as basically isolating the ethernet segment) and using squeezelite to front load network traffic, it seems that ethernet traffic (or processing ethernet traffic) could be causing SQ to degrade.

 

On the other hand, Roon continually streams data to Roon Bridge.  Would a higher priority for ethernet make that interface more predictable/less variable as packets on the ethernet segment continually interrupt the system?

 

The squeezelite buffer experiments and the OG bridged network findings do point to something happening on the ethernet processing side of the end point (which is wild).

[ray-dude]

8 minutes ago, rickca said:

Once you have things working, make a backup of your flash drive.  I had things working just like I wanted.  Then I edited a single file.  The next time I booted I had filesystem integrity issues.  I'm not sure it's recoverable so I'm starting over.  Don't be like me.

 

Better still, write up your recipe and post it here!  

 

Every time someone does, I pick up a new tidbit about how to configure/tune these systems (blacklist.conf....who knew?)

 

 

[ray-dude]

38 minutes ago, greenleo said:

What is the OG report?  Any link?

 

Sorry, I was referring to the original tweak from the mother "A novel way..." thread (OG == Original Gangster ... I blame my teenager for polluting my vocabulary

 

I did start to work my way through the mother thread, but there is a LOT there.  I'm currently holding out for an  EtherGen (or equivalent), but I'm tempted to do some bridging experiments, just for the learning.  I appreciate the guides!

[ray-dude]

Just now, austinpop said:

 

Hi Ray,

 

Yes, as Leo points out, please do use the first post index as a way to make some sense of the OG (I like it!) thread.

 

BTW - how is the squeezeilte parameter optimization going? I still need to try your settings. So is this still the high water mark for you:

1. -b 4097152:97152
2. -a 1000000:4::

?

 

My apologies to all who speak English vs Teenager American (or even Texan   

 

Yes Rajiv, these are my current settings, at least until I have some more time for listening tests.  I thought I was hearing some differences as I went from 1MB to 10MB on the -a parameter, but fatigue was setting in and I wasn't trusting my critical hearing any more.   I need to repeat these test later this week.

 

For the -b parameter, I was arbitrarily shifting as much of the 4GB buffer as felt safe to the streaming side.  I was evaluating order of magnitude impact by testing extremes, so please don't consider this optimized. 

 

As you play with these, I'd be curious to hear your impressions on SQ lift that is still attributable to the server NUC.  I only have a single NUC right now so I haven't been able to experiment, but it seems that if the network activity from server to end point is all in a ~1 second burst, the impact of the upstream server should be diminished for the other 99% of music playback time.  (Per your finding, this would obviously not the case with Roon Bridge on the end point, since the server NUC would be streaming continuously)

 

Basically, do gratuitous buffers on the end point NUC (both to the streaming side and to the DAC side) make the upstream server less critical for SQ?

 

 

[ray-dude]

7 hours ago, rickca said:

@BigAlMc the procedure I used with ImageUSB on a Windows 10 machine is much faster than copying between two USB flash drives on something like NUC7PJYH.  It took about 10 minutes for a 64GB stick.  You just need space available on a hard drive or SSD.  I will report back here once I've created a new USB stick from the .bin file and tested it.  I'm currently waiting for delivery of another 64GB stick.

 

I certainly agree that we need a best practice method of making a backup copy of a working system.

 

I've given up trying to use OSX for maintaining my AudioLinux disks.  I'm by no means a linux guru, but I'm finding the easiest way to do things with AL is to work from within linux itself.

 

For backing up a USB stick, if you boot to ram, you can insert a second USB stick into your NUC and clone your primary stick over to it using the dd command (sticks are $10 so keep lots of backups as you experiment!)

 

Here is a flow with commentary in italics, what I'm typing in bold, what what I'm looking for in bold underline:

 

 

Log in as root (default password is audiolinux0)

 

[audiolinux@audiolinuxquad ~]$ su

Password:

 

 

Confirm that you’re booted into RAM mode

 

[root@audiolinuxquad audiolinux]# ramroot status

:: ramroot status: enabled

:: root filesystem loaded to RAM

 

 

Check the device ID of your primary USB stick (it should be plugged in)

 

[root@audiolinuxquad audiolinux]# fdisk -l

Disk /dev/zram0: 11 GiB, 11775508480 bytes, 2874880 sectors

Units: sectors of 1 * 4096 = 4096 bytes

Sector size (logical/physical): 4096 bytes / 4096 bytes

I/O size (minimum/optimal): 4096 bytes / 4096 bytes

 

 

Disk /dev/sda: 28.7 GiB, 30752000000 bytes, 60062500 sectors

Disk model: Ultra USB 3.0   

Units: sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disklabel type: gpt

Disk identifier: 524A1828-2002-4187-B469-3E7A2DE53ED0

 

Device       Start      End  Sectors  Size Type

/dev/sda1     2048  1001471   999424  488M EFI System

/dev/sda2  1001472 60062466 59060995 28.2G Linux filesystem

 

 

Plug in the USB stick you want to back up to (it should be at least the same size as the primary)

Check the device ID of your target USB stick (it should be plugged in)

 

[root@audiolinuxquad audiolinux]# fdisk -l

Disk /dev/zram0: 11 GiB, 11775508480 bytes, 2874880 sectors

Units: sectors of 1 * 4096 = 4096 bytes

Sector size (logical/physical): 4096 bytes / 4096 bytes

I/O size (minimum/optimal): 4096 bytes / 4096 bytes

 

 

Disk /dev/sda: 28.7 GiB, 30752000000 bytes, 60062500 sectors

Disk model: Ultra USB 3.0   

Units: sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disklabel type: gpt

Disk identifier: 524A1828-2002-4187-B469-3E7A2DE53ED0

 

Device       Start      End  Sectors  Size Type

/dev/sda1     2048  1001471   999424  488M EFI System

/dev/sda2  1001472 60062466 59060995 28.2G Linux filesystem

 

 

Disk /dev/sdb: 28.7 GiB, 30752000000 bytes, 60062500 sectors

Disk model: Ultra USB 3.0   

Units: sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disklabel type: gpt

Disk identifier: 524A1828-2002-4187-B469-3E7A2DE53ED0

 

Device       Start      End  Sectors  Size Type

/dev/sdb1     2048  1001471   999424  488M EFI System

/dev/sdb2  1001472 60062466 59060995 28.2G Linux filesystem

 

 

Based on above, my primary USB stick is /dev/sda and my target backup USB stick is /dev/sdb

Now use the “dd” command to copy the primary to the secondary

Be careful what you type! You’re running as root and can wipe things out with a typo

You’ll see progress status, takes about 23 minutes for me to dupe a 32GB stick

 

[root@audiolinuxquad audiolinux]# dd if=/dev/sda of=/dev/sdb bs=4M status=progress

30752000000 bytes (31 GB, 29 GiB) copied, 1328 s, 23.2 MB/s

7331+1 records in

7331+1 records out

30752000000 bytes (31 GB, 29 GiB) copied, 1399.85 s, 22.0 MB/s

 

 

Remove your primary USB stick, and reboot the system off the backup to test it

 

[root@audiolinuxquad audiolinux]# reboot

 

 

If you want to create an .img file, you need to have a remote file server mounted.  That will let you dd to a file on the remote file system (substitute the path to your mounted file system in the command below).

Use caution and make sure you're copying to the right location! You can wipe out drives with typos when using "dd"

 

[root@audiolinuxquad audiolinux]# dd if=/dev/sda of=/path/to/file.img bs=4M status=progress

 

 

 

 

 

 

[ray-dude]

Stepping back a bit, I would encourage linux-phobes (like me) to take AudioLinux as an opportunity to learn how linux works and poke under the covers.  Piero is doing a fantastic job hiding the messy internals, but a wonderful thing about his work is that it is all there to take a look at learn what is going on behind the scenes.  I'm certainly learning a lot about linux as I fiddle with AudioLinux.

 

For linux newbies, here is a walk through to get you started poking around (with apologies to the linux pros):

 

For example, what happens when you save your system when you're in RAM mode?  Well, the first thing to sort out is what happens when you type "menu"

 

Using the command "whereis" you can see where the menu script is located:

 

[root@audiolinuxCel audiolinux]# whereis menu

menu: /usr/bin/menu /usr/include/menu.h /usr/share/man/man3/menu.3x.gz

 

 

You can then look at this script and what commands it is executing behind the scenes.  For example, what happens when you select "(1) Configuration"?

 

[root@audiolinuxCel audiolinux]# more /usr/bin/menu

 

.

.

.

case $CHOICE in

        1)

          alconf

          menu

            ;;

.

.

.

 

It runs a script called "alconf".  So what happens when you run "(23) Save system (if in RAM mode)"?

 

 

[root@audiolinuxCel audiolinux]# whereis alconf

alconf: /usr/bin/alconf

[root@audiolinuxCel audiolinux]# more /usr/bin/alconf

.

.

.

            22)

           wait

          display_result "$(sudo ramsave)"

          menu

            ;;

.

.

.

 

It executes a script calls "ramsave".  You can now execute ramsave from the command line yourself, or look in the ramsave script to see all the goodies that are happening there.

 

Piero has very well structured scripts, so they are very easy to follow.  I'm certainly enjoying the linux primer by following his work.

 

 

 

[ray-dude]

7 hours ago, rickca said:

@ray-dude thank you so much for taking the time to document those procedures in detail.  It's extremely helpful.

I was just thinking that it would be useful to find out exactly what various menu items do under the covers.  It's a very good way to learn.

 

I should have mentioned it (for future folks searching the thread), but the "what does ramsave do" example I gave wasn't an idle one.  I've been experimenting with having my music library in memory on my 7i7DNKE and lost it after an upgrade/reboot (huh).  

 

Turns out I had put the local library in /media/music (adjacent to /media/samba, which is where I mounted my music share from my Mac mini server).  Diving into the ramsave script, I saw that the /media directory is excluded from the rsync process that copies things back to the USB stick.

 

Moved my local music directory to the root directory, and problem solved (without having to pester Piero

 

[ray-dude]

2 hours ago, rickca said:

@ray-dude there are tons of Linux primers out there.  Is there one you recommend to more fully understand what is going on under the covers of the menu items?  I don't want a reference 'bible' kind of book, but rather one that highlights the commands and procedures we are likely to use with AL without getting too sophisticated.  I have a good background in programming (started with 370 assembler and principles of operation).  I even taught undergrad Computer Science at a university.  But that was lightyears ago.

 

I'm sure there are but I haven't looked for one.  I have a basic Unix background from 30 years ago (yikes!), so I can suss out man pages and unwind shell scripts.  For me everything else has been google, and trying stuff at the command line (and keeping a cheat sheet up to date!)

 

By convention, most Unix apps have a -h (help) flag (for example "squeezelite -h"), and/or a man (manual) page (for example, "man dd").  Most of the visible things Piero has done with AL has been with shell scripts, so you can use "more" or your text editor of choice (I use nano) to look at files.

 

A bonus when you're running in ram mode and have the USB stick removed: no matter how badly you mess up, you're a reboot away from a clean start!

 

[ray-dude]

Quick question for those that have been playing with LMS as their music server.  I'm finding it plays much nicer with the squeezelite end point vs Roon Server, but I would like to do an apples to apples comparison.

 

Which processes are folks singling out to include in /opt/rtapp/rtapp.conf to give the server process realtime priority?  It seems to be running as a daemon, with a lot of subprocesses,, but I'm getting a bit lost tracking them down to see which should have a priority boost.

 

 

[ray-dude]

Thank you Rajiv! I was assuming the Perl script was invoking some binary process that actually connected to the end point.  Back in the day, Perl was a write once, rewrite many scripting language for me, so I twitch a bit whenever I have to figure out someone else’s Perl code   I may have some leftover eggnog tomorrow and see if there is a gem in there that actually manages the interface with the end point. 

 

 

[ray-dude]

17 hours ago, ray-dude said:

Thank you Rajiv! I was assuming the Perl script was invoking some binary process that actually connected to the end point.  Back in the day, Perl was a write once, rewrite many scripting language for me, so I twitch a bit whenever I have to figure out someone else’s Perl code   I may have some leftover eggnog tomorrow and see if there is a gem in there that actually manages the interface with the end point. 

 

 

 

 

I was able to spend some time this morning walking through the LMS source tree.  The transport is a lot simpler than I thought it would be (Roon-think had polluted my assumptions).  Looks to be a very simple HTTP streaming transport, backed by more sophisticated perl-based state machine for managing what is getting streamed (seeking in files, pause, play, etc).

 

The only items I could see that may need higher priority were some of the transcoding binaries (faad, flac, etc) in /opt/logitechmediaserver-git/Bin/i386-linux   However, for server-based sources (vs streaming sources from Tidal or Qobuz), things are getting streamed so quickly to the large buffer squeezelite end point (-b and -a flags), I'm not sure that fiddling with transcoding priorities would do much to move the needle (it just takes a second or two to move things to the end point)

 

In fact, as a neat parlor trick, if you kill LMS on the server (or Roon server), the end point happily keeps on playing, with no change in SQ that I could detect.  Makes me wonder again if any server side optimizations have an impact once you get past the initial 1-2 seconds it needs to fill the gratuitously large endpoint buffer.

 

EDIT: same trick works if you disconnect the end point from ethernet after the buffer has been loaded up...the end point just happily keeps playing (a brute force way to avoid the skipping you get with Roon server

 

@austinpop when you were doing your servers and storage evaluations, were you using the large buffer(s) (-b and -a) squeezelite configuration on your end point?  If so, were you able to hear a difference in SQ?

 

 

 

[ray-dude]

 

I was able to do some more experiments playing with gratuitously large end-point buffers, with some interesting findings.

 

As a reminder, I’m running AL headless as a RAM boot server on a 16GB NUC7i7DNKE, and my end point on a headless AL RAM boot 8GB NUC7CJYH.  End point is connected by USB to a Chord stack.

 

For the NUC server, I’m running both Roon Server and LMS, testing a combination of local music files (in ram), SMB mounted files from my Mac Mini server (vanilla ethernet/switches/server) and TIDAL streaming

 

For the end point, I’ve been experimenting with large buffers in squeezelite (configuration below)

 

/usr/bin/squeezelite -o front:CARD=SCALER,DEV=0 -b 4097152:97152 -d all=info -a 50000000:4 -c pcm -x -p 93

 

With this config, music is loaded from the server to the end point in a couple seconds.

 

 

The hypothesis I’ve been chasing is that a large in memory buffer (-b parameter) between the end point and the server will insulate the end point from any upstream fun, and a large in memory buffer between the endpoint software and the DAC (-a parameter) will insulate SQ (as much as possible) from other things going on on the end point.

 

What I’ve been playing with today is trying to understand how the buffer is loaded up by the server in various configurations.  To keep things consistent and avoid misinterpreting things, I do the following:

 

* Kill and restart squeezelite on the end point (to clear the buffers, etc)

* Start up playback on the server, queuing up multiple tracks of the same type (TIDAL, local, file server, bitrate, etc)

* Wait ~10 seconds, then physically disconnect the end point from ethernet

* All content was 44/16 lossless, so  I am long way from filling any buffers

 

TL/DR, with the squeezelite endpoint, Roon Server and LMS both buffer up current and next track, whether local files or streaming TIDAL content.

 

 

 

 

Roon Server, TIDAL content

 

Roon seems to buffer up the current track and the next track.  With longer classical pieces, I’ve had the end point happily keep playing for 20+ minutes when disconnected from ethernet (I have not tested the limit of disconnected playback)

 

 

Roon Server, local content (on NUC)

 

Same as above

 

 

Roon Server, content from file server (Mac Mini)

 

Same as above

 

 

LMS, TIDAL content

 

I haven’t gotten this to work yet, so not tested

 

 

LMS, local content (on NUC)

 

LMS also seems to queue up the current track and next track into the end point buffer.

 

 

LMS, content from file server (Mac Mini)

 

Same as above

 

 

 

 

Quick Thoughts

 

I did not hear any SQ difference when disconnecting the end point from ethernet.  I was physically getting up to do so (vs having someone else do so while listening) so this isn’t conclusive, but I also did not hear any difference in SQ when I killed the server process (Roon Server and LMS) on the server NUC (caveat: I have vanilla power and network in my setup)

 

I found it very interesting and encouraging that Roon is doing a stream-ahead preload of the buffers with TIDAL (buffering will give the same advantage with TIDAL as it does with local files)  There may be a practical path to effectively decouple the end point from the upstream part of your chain (except for the couple seconds it takes to buffer up content from the server)

 

In a buffer heavy end point configuration, it may be most beneficial to make the NUC7i7DNKE the end point (improves end point SQ) and make the NUC7CJYH the server (since it does very little when not filling up the buffer)

 

There is more experimentation to do in more normal operations (when does buffer get refilled, what happens when skipping songs or seeking within a song, etc)

 

With access to both the squeezelite and LMS source code, there may be options to further optimize/batch/minimize network interactions between server and end point (the SQ lift by having network traffic front loaded is material....seems like idle ethernet == good for the end point)

 

 

 

 

[ray-dude]

12 minutes ago, austinpop said:

 

I just posted my impressions on server differences over on the new thread Larry created. But to be clear, I was using Roon Bridge in those comparisons. I could not rapidly switch between 2 different instances of Roon Server and have it work with a Squeezelite endpoint.

 

If what you suggest is true, then if I redid the experiment by running LMS on both my servers, and switching back and forth by attaching the squeezebox endpoint to one or the other, I should perceive no difference between the servers?

 

I will try this later tonight if possible.

 

Yes, that would test the hypothesis.

 

In watching the squeezelite logs, it looks like it tries to reconnect to the original server for 30 seconds before going back into discovery mode.  To switch quickly, you may need to kill the end point, stop Roon Server or LMS on server #1, bring up Server #2, and restart squeezelite and have it reconnect to your new server

 

It would be fantastic if large end point buffers give some decoupling from the server configurations (I'm reading your post now...lots to digest there)

[ray-dude]

Rajiv, my critical listening stamina is (way) at an end for the day, but I ended up with LMS on both my server NUC (all music files local and in RAM) and LMS on my MacBook Pro (WiFi, accessing files from my Mac Mini file server)  These are basically my best/worst case configs, without having to worry about Roon authorization switching.

 

To make swapping back and forth easier on the end point, I'm specifying the server IP in the command line for squeezelite:

 

/usr/bin/squeezelite -s 192.168.4.126 -o front:CARD=SCALER,DEV=0 -b 4097152:97152 -d all=debug -a 50000000:4 -c pcm -x -p 93

 

Lets me swap back and forth in the start up time for squeezelite (<10 seconds).

 

There is enough network chatter going back and forth between server and endpoint (status stuff) that I'm also experimenting with kill LMS on the NUC and LMS on the MacBook Pro to see if I can detect any delta in SQ.

 

My ears are cooked, so I'm going to revisit things tomorrow with a fresh set of ears.  Looking forward to hearing about what you're hearing!

 

 

[ray-dude]

A quick update for this thread on my experiments with large buffer squeezelite end points.  When running LMS and Roon Core on various servers, I wasn't able to detect any difference in SQ when connecting to my large buffer squeezelite NUC core.  Similarly, I couldn't detect a difference in SQ for files local (in RAM) on my NUC server vs files pulled from an SMB share over the network.  

 

I'm looking forward to hearing reports from folks with more power and network configurations, in case they are able to pick up on differences that I can not.

 

Those interested in the discussion should check out the server thread (linked below)

 

Best wishes for the New Year!

 

Ray

 

 )