Article: Streaming & Local Content Through HQPlayer - A Nice Alternative With The sonicTransporter

[Miska]

5 hours ago, Cebolla said:

It appears that the MinimServer UPnP media server isn't quick enough. 

 

Another thing to note is that when HQPlayer makes GET request to the UPnP server, it fetches about 10 seconds worth of audio to pre-buffer. This should fit well within one second time window with current 1, 2.5 or 10 Gbps networks.

 

[Miska]

8 hours ago, bibo01 said:

@stefano_mbp@The Computer Audiophile @Miska I am still unclear about this UPnP business...

Miska, is it fair to say that HQPlayer is somehow at the mercy of Rygel in terms of receiving UPnP commands for rendering?

 

Yes, Rygel is the UPnP stack and controls HQPlayer based on the UPnP commands.

 

8 hours ago, bibo01 said:

If that is the case, there must be some miscommunication between server-controller-Rygel in some particular circumstances that cause delay.

 

That is likely the cause.

 

8 hours ago, bibo01 said:

If, as Simon/MinimServer poited out, the delay is in the Rygel UPnP stack, is it possible to propose solutions to Rygel's main development group (Zeeshan Ali, Jens Georg, Thijs Vermeir, James Henstridge and Jussi Kukkonen) in the GNOME git repository?  (There is a "Jussi" there too )

 

That could be one way. I know Zeeshan and (the other) Jussi personally, they are my ex-colleagues from my Nokia days.

 

Rygel itself was used for example in our N9 phone and was DLNA certified already at that point. (there are many UPnP implementations talking about DLNA, but only few of those have been certified). Note that HQPlayer is not DLNA compliant, as it doesn't support 128 kbps CBR MP3 or WMA which for example are requirements for DLNA.

 

[Miska]

5 hours ago, stefano_mbp said:

• this playlist is then sent to the renderer and at this point the control point task ends,
• it is in fact the renderer who “takes the reins of the game” and takes care of asking Minimserver for the songs to be played.

 

No, this would be OpenHome it is not part of UPnP specification. And how for example HQPDcontrol works with HQPlayer server using HQPlayer's control protocol.

 

But not UPnP AV. In UPnP AV, Control Point holds the playlist and instructs renderer to proceed to the next track when the previous track ends. This is also similar to how Roon uses HQPlayer.

 

The optional extra feature in UPnP AV, SetNextTransportURI allows Control Point to provide upcoming next URI to the Renderer before the current track ends.

 

UPnP Renderer then issues HTTP GET request for the content to the Media Server.

 

5 hours ago, stefano_mbp said:

In other words, Roon operates in "push" mode against HQPlayer while the UPNP renderer operates in "pull" mode against Minimserver.

 

No, both work in the same way.

 

[Miska]

5 hours ago, stefano_mbp said:

The problem is that an UPNP server is passive, it waits for a request from the renderer, otherwise it wouldn't be an upnp server.

 

UPnP Media Server is passive. But UPnP Control Point is active.

 

When HQPlayer performs GET request to the UPnP Media Server, it assumes to get ~10 seconds worth of audio instantly. If Media Server at this points starts doing something like calling ffmpeg to transcode content or otherwise lags in being able to provide 10 seconds of audio, you may get a gap because HQPlayer runs out of content to play.

 

[Miska]

12 minutes ago, stefano_mbp said:

I would like to have your comment about Simon Nash answer about this issue:

”In my test environment (RPi3 for server and RPi4 for renderer, 16/44 FLAC files, no transcoding), the great majority of the delay between tracks that causes these gaps is within HQPlayer (specifically, within the Rygel UPnP stack that HQPlayer uses to enable it to act as a UPnP renderer). Because HQPlayer doesn't use the UPnP gapless protocol, any significant delay during track changing breaks gapless playback.
If HQPlayer is running on a faster machine, renderer track-changing delay will be reduced and any server delay will come into play. Increased resolution and/or server-side transcoding will increase server delay.
The only way to make this work correctly in all cases is for HQPlayer to implement the UPnP gapless protocol. This enables the renderer to start streaming the next track before the current track has finished playing.”

 

Rygel does support SetNextTransportURI.

 

HQPlayer doesn't implement any UPnP stuff at all. Only Rygel's renderer-interface.

 

But the way HQPlayer works, if the control point and media server are quick enough, you get gapless playback even without SetNextTransportURI, because there's enough buffer.

 

[Miska]

5 hours ago, bibo01 said:

Then...can you please talk to them?! 

 

About what? That some third party software is not behaving nicely with their software?

[Miska]

1 hour ago, bibo01 said:

I do not have a technical solution, but given that there is a miscommunication between server-controller-Rygel, I am sure you might have a good idea on what to propose in order to avoid a delay in Rygel's UPnP stack.  

 

Or maybe the problem is in the server / control point implementation, because the problem does not appear with all implementations.

 

For me, this works well enough for playing Tidal. I don't see point in playing local content using UPnP, since HQPlayer has it's own library and control application infrastructure.

 

[Miska]

1 hour ago, stefano_mbp said:

- browsing library by metadata is much more effective and pleasant with Minimserver

 

This depends entirely on the control application implementation. With HQPlayer library, control application gets the entire library data and can present it any way it likes. This is also the way HQPlayer Client does it by offering four different views to the same library data.

 

1 hour ago, stefano_mbp said:

- my library, together with flac and dsf, includes many thousands Alac files incompatible with HQPe, the conversion is not feasible but I can transcode them on the fly to wav with Minimserver

 

This is certainly one gapless problem, because transcoding adds lag to the file delivery. Especially because I think MinimServer calls ffmpeg binary to convert entire file at once when the file is requested, instead of using libavformat/libavcodec to do it as a stream. There is no guarantee how much time it takes to transcode a file. In any case this is potentially one of the core reasons why it is not working.

 

1 hour ago, stefano_mbp said:

... the correct functioning of the gapless UPNP would solve all these aspects perfectly, would also constitute a more than valid and interesting alternative to Roon, but I see, very sharply, that unfortunately there is no interest in meeting the customer's needs ...

 

Yeah, I cannot force MinimServer developer to figure out why it's slower than some other servers. Nor Rygel developers either to modify their implementation.

 

[Miska]

9 minutes ago, stefano_mbp said:

Broken gapless occurs even with not transcoded flac as Simon Nash pointed out

 

Here's playing tracks on Atom category CPU, from MiniDLNA server running on very old Core i5 NUC, using mConnect Player as UPnP Control Point on my Nokia 8.1 5G phone:

** Message: 12:52:18.319: idle_req
** Message: 12:52:18.409: stop notify
** Message: 12:52:18.564: state -> STOPPED
** Message: 12:52:18.612: state -> STOPPED
** Message: 12:52:18.625: state -> STOPPED
** Message: 12:52:18.647: uri -> http://192.168.71.14:8200/MediaItems/4452.flac
** Message: 12:52:18.740: state -> PLAYING

 

You can see it is less than 500 ms even on this slow CPU while doing PCM upsampling.

 

9 minutes ago, stefano_mbp said:

There is no “intelligent browsing” at all available in HQP … trust me

For your reference https://minimserver.com/features.html

 

Browsing is feature of the control client. Not HQPlayer server. HQPlayer server just provides library data to the client and it is up to client to figure out how to distill "intelligent browsing" from the content metadata.

 

HQPlayer is not limited in the way UPnP is, that clients would present things the way server exposes them.

 

[Miska]

13 minutes ago, stefano_mbp said:

…  could be true (with many doubts) but at the moment no app is capable of this functionality with HQP.

This functionality is already available instead with Lumïn, Linn, Kazoo, MConnect, Bubbleupnp, Naim, Lightning DS and more using Minimserver as media server, even in LMS and Daphile using Remote Library plugin … 

 

Yes of course if the UPnP Control Point app just presents what the server sends instead of distilling it. Roon and HQPlayer don't operate such way.

 

[Miska]

On 10/20/2021 at 2:55 PM, Cebolla said:

Instead of the 1s fixed time, how about allowing the user to set the UPnP end of track time adjustment, say from a range 1s -10s default 1s (assuming you have enough buffer, or can increase the buffer to accommodate, if user unnecessarily selects 10s in quick control point/media server case)?

 

Technically, it would only need adding a configuration option(s). But on the other hand it is band-aid for something that should work without. I'm just always concerned about number of configuration options, I already get so much complaints about configuration being too complex. At the moment there are three internal parameters that control this behavior.

 

[Miska]

2 hours ago, Geoffrey Armstrong said:

Actually, I have tested my sw in the past with embedded and it did work. I just need to point out though that in the most recent version of HQPlayer, Jussi has introduced an encryption feature for communication between HQPlayer and other apps. This is a great feature, which I asked for myself. Unfortunately though, I have not had time to update any of my apps to take advantage of it yet. Because of that, if anyone's trying to use newer versions of HQPlayer with this feature, they probably won't work with HQAV. Perhaps @miska you could just post here from which version HQPlayer incorporates the encryption feature? My sw will only work with versions prior to that.

 

It shouldn't break existing control applications, so it is kind of optional, for now. But recent HQPlayer Client for example requires it to operate.

 

[Miska]

15 hours ago, Miska said:

Technically, it would only need adding a configuration option(s). But on the other hand it is band-aid for something that should work without. I'm just always concerned about number of configuration options, I already get so much complaints about configuration being too complex. At the moment there are three internal parameters that control this behavior.

 

I've added environment variable options to override default buffering values. This way it won't pollute regular settings. It will come out in next release. Three values (no need to set all):

1. HQPLAYER_BUFFER_TIME to set amount of buffer (in ms, must be multiple of 100 ms)
2. HQPLAYER_IDLE_MARGIN to set how much margin there is left before feed is needed (in ms, must be multiple of 100 ms)
3. HQPLAYER_IDLE_TIME to set how long to run idle before stopping

 

You can set these in /etc/default/hqplayerd

 

[Miska]

42 minutes ago, bibo01 said:

Can you please give us some indications on how to modify them - how they relate to each other, what is default value, etc...

 

Think BUFFER_TIME as size of a whisky container. There's a valve at the bottom to let whisky run out at constant speed. New whisky is added on top. You can also add just water. But liquid should be kept running always. IDLE_MARGIN is threshold line telling how empty the container can run before you must start adding either more whisky or just water, or otherwise you risk running out of liquid.

 

IDLE_TIME tells how long the liquid is kept running when only water is being added, before the output valve is closed because it seems like we are out of whisky and there's only water left.

 

For the sake of clarity we assume the liquid doesn't get mixed when we add water on top. Whisky is music, water is silence.

 

P.S. I have made some fixes and now SetNextTransportURI also seems to work as intended (at least with mConnect Player). Now the track transition time with Tidal streams is less than 100 ms on Atom CPU. New point release coming shortly.

 

[Miska]

1 hour ago, Miska said:

P.S. I have made some fixes and now SetNextTransportURI also seems to work as intended (at least with mConnect Player). Now the track transition time with Tidal streams is less than 100 ms on Atom CPU. New point release coming shortly.

 

There are now Release Candidate HQPlayer OS images of 4.26.1 available for download. This includes both the additional control variables and fixed SetNextTransportURI support (related to Rygel - HQPlayer communication).

 

In mConnect Player, remember to enable "Gapless support" from the menu.

 

If this seems proper, I will make build packages for regular distributions shortly and make a new complete release.

 

[Miska]

2 hours ago, stefano_mbp said:

• DSD as Direct SDM and SDM output mode checked (then bit perfect): gapless not working. DSD files served by Minimserver without any transcoding.

 

This is one area where you would win by using HQPlayer to play your album in album mode instead of playlist mode.

 

When you play through UPnP, HQPlayer doesn't know if subsequent tracks belong together or if they are separate items, so they are assumed to be separate unrelated items.

 

There's a "state reset" processing for unrelated DSD tracks in HQPlayer to reduce amount of pop/click you get when transitioning between tracks. I have now uploaded updated build where you can disable this by setting HQPLAYER_RESET_SDM=0 in /etc/default/hqplayerd. Now you get gapless playback for related DSD tracks, but louder pop/click when transitioning between unrelated tracks. Only solution to this is to play natively from HQPlayer's library.

 

2 hours ago, stefano_mbp said:

I would have liked to modify the additional control variables (to try if it could solve the Direct SDM  DSD issue) but I cannot find 

/etc/default/hqplayerd 

 

That file doesn't exist by default since it would be empty, so you can just create it. systemd will read it on next restart (just reboot your server).

 

[Miska]

1 hour ago, bibo01 said:

Would it be possible for the server to send the information in album mode, if that feature was added to the server side? Can album mode be understood by the UPnP chain? 

 

No, it is not part of UPnP specification. But you can naturally switch over from UPnP protocol to HQPlayer's control protocol and then you don't have such limitations. Then you have also OpenHome style capabilities with server-side playlists and much much more.

 

[Miska]

4 hours ago, bibo01 said:

Trying to understand.... Are you saying that it can only be achieved using HQPlayer's control protocol (like Roon) which "mimics" OpenHome style capabilities?

 

No, Roon isn't doing that since Roon isn't using HQPlayer's library. Roon operates in a different way with HQPlayer (using HQPlayer's control protocol though).

 

4 hours ago, bibo01 said:

Can album mode be achieved with an OpenHome media server like BubbleUPnP server? And is Embedded compatible as OpenHome renderer?

 

No, it cannot. And no it is not compatible with OpenHome.

 

By design UPnP is split into three parts, media server, control point and renderer. Media server holds the content directory and would understand content relationships. Control point tells renderer what track URLs to play from media server, when it is time to play those. But already control point is lost on the real content relationships, instead it sees abstraction created by the media server. Renderer only sees one meaningless URL at a time.

 

[Miska]

27 minutes ago, stefano_mbp said:

I am tempted to try to change the other environment variables but I fear they may have a negative impact on the functioning of the gapless for PCM files which is now perfect. What do you think?

 

Likely it won't help at all on this matter.

 

[Miska]

OTOH, the way native HQPlayer works is that HQPlayer holds the library. And the player engine. Then control client (HQPlayer Client, HQPDcontrol, HQPWV, etc) can tell HQPlayer to load and play a specific album from the library. Not some list of tracks, but album (album mode). Or alternatively some arbitrary set of tracks from the library (playlist mode). You can also ask HQPlayer to play from alternative sources, such as CD disc, internet streams, analog or digital inputs, USB input...

 

[Miska]

9 hours ago, stefano_mbp said:

… I tried it but it doesn’t make any difference, I suppose this is due to “related tracks” that cannot be understood using UPNP.

I am tempted to try to change the other environment variables but I fear they may have a negative impact on the functioning of the gapless for PCM files which is now perfect. What do you think?

 

I just uploaded updated images of 4.26.2 with a fix for one regression, and /etc/default/hqplayerd wasn't enabled in the systemd service-file, so the setting didn't have effect. Please download the updated image and see if it now works.

 

[Miska]

1 hour ago, stefano_mbp said:

playing through UPNP “chain” nothing changes (with or w/o setting HQPLAYER_RESET_SDM=0) , there is a very long delay between tracks, variable from 5 up to 9 seconds …

 

That is strange, sounds like there is something happening on the path, at the media server side maybe?

 

You could also try setting HQPLAYER_STREAM_FREEWHEEL=1 which will fetch the entire track as quickly as possible. Not great if you are on a limited internet connection, but in local network should be less of an issue, if the network supports QoS properly. Otherwise it may cause drop-outs on NAA.

 

[Miska]

11 minutes ago, The Computer Audiophile said:

I've run into some playback stoppages, but I don't have anything definitive for troubleshooting or even know if it's something strange going on here. 

 

4.26.2 I released today fixes those