Truncating MQA files to 16 bits and the blue light still shines

[mansr]

6 minutes ago, rickca said:

Is this your version of show me the money?

We've already seen the money, or lack thereof.

[FredericV]

Some updates. Our member "evangelist" was trying to attack my research as a being fake / gossip:

 

Sorry, but I still have a good old 2.2 mytek firmware which does not lie, and this firmware shows a dark grey MQA logo when non-MQA is played, and bright blue MQA logo when MQA is detected. It has no blue dot on the right.

So we updated to the latest version of Mytek's firmware, and now to my surprise the following happens:

- when playing the unstripped 24/44.1 MQA file, it shows 24/352.8 Khz and the blue MQA dot:

 

- when playing the stripped version with 8 LSB bits stripped, it will now claim it's still 24/352.8



 

The hint this is a lie, is because our 10 second loop interrupts the MQA markers for a short while, so for a fraction of a second, the DAC goes back to the actual input resolution of 16/44.1:

 

I modified my script to allow more bits to be blanked, so when stripping 9 bits, the 15 bits content in a 24 bit WAV container is still showing 16/44.1 on the Mytek. I wanted to be sure that the Mytek still detects bit blanking or padding for non-MQA, it still does work as expected for non-MQA.

So if the real MQA file, and stripped version with only 16 bits of entropy, both show decoding to 24/352.8, how does the customer know he is not being scammed?
 

[mansr]

3 minutes ago, FredericV said:

So if the real MQA file, and stripped version with only 16 bits of entropy, both show decoding to 24/352.8, how does the customer know he is not being scammed?

Easy, if the blue light is on, it's a scam.

[mcgillroy]

Anybody chipping in for a bunch of flowers to be send to MQAs office?! I think they deserve some!

[MarkS]

2 hours ago, mansr said:

Easy, if the blue light is on, it's a scam.

 

Funny response,  but this seems like a legit major issue for MQA.

[FredericV]

More fun: replacing the first unfold data with a repeating ASCII string "MQASHILL" still authenticates the file.

More experiments. Small correction: when 9 bits are stripped, the Mytek actually detects 15 bits (and not 16 bits). I looked at a distance and 15 and 16 look too similar. But here it is:




We improved the script to allow extra tampering with MQA files. We provide the 10 second sample files for each manipulation:

1. input.wav

This is 10 seconds from 2L-050_01_stereo_DXD_WAV.mqa.flac which mainly contains noise, and the first notes from the track.
This one is not manipulated.

2. mqashill.wav

 

Same file, but with the 8 LSB bits replaced by repeating ASCII string "MQASHILL". This tampered file still authenticates, and decodes to 24/352.8 on the mytek.

This was created with a newer version of my script which allows more tampering, not yet released:

./mqastripper.pl -i input.wav -o mqashill.wav -l 10 -t -d
Input file details: $VAR1 = {
          'sample_rate' => 44100,
          'total_length' => 48090204,
          'block_align' => 6,
          'length' => '181.746666666667',
          'data_start' => 44,
          'bits_sample' => 24,
          'channels' => 2,
          'wave-ex' => 0,
          'data_length' => 48090168,
          'bytes_sec' => 264600,
          'data_finish' => 48090212
        };
Samples: 8015028
Rate: 44100
Bits to shift = 8
-127,40 -> -179,77
LSB LEFT  = 77 M
LSB RIGHT = 77 M
451,304 -> 337,337
LSB LEFT  = 81 Q
LSB RIGHT = 81 Q
178,-169 -> 65,-191
LSB LEFT  = 65 A
LSB RIGHT = 65 A
89,203 -> 83,83
LSB LEFT  = 83 S
LSB RIGHT = 83 S
444,163 -> 328,72
LSB LEFT  = 72 H
LSB RIGHT = 72 H
-210,-97 -> -183,-183
LSB LEFT  = 73 I
LSB RIGHT = 73 I
493,518 -> 332,588
LSB LEFT  = 76 L
LSB RIGHT = 76 L
136,-585 -> 76,-692
LSB LEFT  = 76 L
LSB RIGHT = 76 L
139,769 -> 77,845
LSB LEFT  = 77 M
LSB RIGHT = 77 M
735,-545 -> 593,-687
LSB LEFT  = 81 Q
LSB RIGHT = 81 Q
-521,657 -> -703,577
LSB LEFT  = 65 A
LSB RIGHT = 65 A

 

3. output-stripped-8.wav

 

This is the 24 bit input.wav, with 8 LSB bits stripped per sample, in a 24 bit container. So 16 bits of entropy at best.
Authenticates and dac claims this is 24/352.8

 

4. output-stripped-9.wav

 

This is the 24 bit input.wav, with 9 LSB bits stripped per sample, in a 24 bit container.
This actually destroys the MQA control data. DAC says no-MQA and 15 bits of data.

So to conclude:

- 24 bit MQA file original authenticates (what else did you expect)
- replacing 8 LSB bits of each 24 bit sample by a repeating ASCII string "MQASHILL" and the file still authenticates and renders 24/352.8 in an MQA dac
- blanking 8 LSB bits and the file still authenticates and renders 24/352.8 in an MQA dac

I did no try to listen to the potential garbage that would come out of mqashill.wav when played on an MQA dac, as this file has new fake unfold content, and still authenticates.

[beetlemania]

13 minutes ago, FredericV said:

replacing the first unfold data with a repeating ASCII string "MQASHILL" still authenticates the file

 Perfectly sensible given that the blue light is foundation upon which MQA stands

[FredericV]

My biggest concern:

MQA always said tampering with the file no longer authenticates the file. We have proven the opposite.

The customer can't differentiate between the following 3 cases which all authenticate:

1. 24 bits original MQA file is playing, decoder in DAC doing 1e unfold + upsampling.
2. truncated version to 16 bits is playing, not doing 1e unfold as the origami data to do this is gone, so only upsampling
3. replacing origami data by garbage, so curious what will be in the ultrasonics, would need to measure this at the output of the Mytek

So authentication is quack on several levels:

- not always authenticated by the mastering engineer who did not give permission, but MQA claims this is the master, as encoded by request of some other party not being the mastering engineer

- fake authentication of tampered MQA files still showing the blue light

[mansr]

What happens if you mess with bit 9, just above the MQA control stream?

[FredericV]

1 minute ago, mansr said:

What happens if you mess with bit 9, just above the MQA control stream?

Coding as we speak.

BTW Hans Beekhuyzen claims the following:
 

Quote

However, if the file is tampered with in any way, the decoder will notice this and will not light that indicator.

Maybe he should read this post. His information is not accurate.

[GUTB]

Doesn't the 8 LSB just contain the ultrasonic unfold data?

[FredericV]

Setting the 9th bit (starting counting from bit 0) via one of these 2 Perl operators, de-authenticates the file, and mytek says it's 24/44.1:
 

Method 1:

$ln=$l | 0b1000000000;
$rn=$r | 0b1000000000;

Method 2:

$ln=$l | 512;
$rn=$r | 512;

 

[rickca]

1 hour ago, FredericV said:

I did no try to listen to the potential garbage that would come out of mqashill.wav when played on an MQA dac, as this file has new fake unfold content, and still authenticates.

I think we know what an MQAshill sounds like.

[mansr]

2 hours ago, GUTB said:

Doesn't the 8 LSB just contain the ultrasonic unfold data?

Yes, so?

[mansr]

1 hour ago, FredericV said:

Setting the 9th bit (starting counting from bit 0) via one of these 2 Perl operators, de-authenticates the file, and mytek says it's 24/44.1:

As expected. The authentication covers only the PCM part of the stream.

 

Here's something else to try. Toggle bit 9 in one sample every few seconds.

[FredericV]

34 minutes ago, mansr said:

As expected. The authentication covers only the PCM part of the stream.

 

Here's something else to try. Toggle bit 9 in one sample every few seconds.

Now it flips the 9th bit every 3 seconds.

pi@mqb:~/quack $ ./mqabit9.pl -i input.wav -o corrupt_every_3secs.wav  -l 30
Input file details: $VAR1 = {
          'bits_sample' => 24,
          'data_length' => 48090168,
          'channels' => 2,
          'wave-ex' => 0,
          'data_start' => 44,
          'length' => '181.746666666667',
          'sample_rate' => 44100,
          'bytes_sec' => 264600,
          'data_finish' => 48090212,
          'total_length' => 48090204,
          'block_align' => 6
        };
Samples: 8015028
Rate: 44100
Sample 0 corrupted
***Sample 132300 corrupted
***Sample 264600 corrupted
***Sample 396900 corrupted
***Sample 529200 corrupted
***Sample 661500 corrupted
***Sample 793800 corrupted
***Sample 926100 corrupted
***Sample 1058400 corrupted
***Sample 1190700 corrupted
***

It ignores the first corrupt sample, blue light almost immediately shines, but after 3 seconds, the light goes out and stays out. It does not recover as long as the same file is playing. I actually have to switch to another MQA file to get the blue light back.

Curious if we start to flip bits in the MQA control stream ....

[mansr]

2 minutes ago, FredericV said:

It ignores the first corrupt sample, blue light almost immediately shines, but after 3 seconds, the light goes out and stays out. As long as there's a datastream without pauze, it does not recover from the corrupt bit.

Use "mqascan -p4" on the file to check the signature packet interval. Then corrupt the PCM data in only some of the authentication blocks.

 

2 minutes ago, FredericV said:

Curious if we start to flip bits in the MQA control stream ....

The control stream has a packet structure, each having a 4-bit checksum. If this doesn't match, decoding is disabled until the next sync pattern is found. If the checksum matches, any corruption will still fail authentication.

[FredericV]

Corrupting the control bit (counting from 0, bit 8) is already detected when the first sample was corrupted, contrary to corrupting the bit just above it (counting from 0, bit 9). This confirm the initial part of the track is not played in MQA.

When I corrupt the control bit after 3 seconds (and not starting with the first sample), and repeat the process every 3 seconds, as soon as it detects a corrupt bit in the control stream, the MQA light goes out, and does not recover.

[jasenj1]

Thank you for your work writing this code and doing these tests.

 

Hopefully, some folks with other MQA DACs can run the files through them and see if there is any difference.

[miguelito]

On 2/22/2018 at 10:45 AM, mansr said:

Easy, if the blue light is on, it's a scam.

Delicious... 

 

It is truly embarrasing that a 16 bit file will trigger the “blue dot of cash” (cash for MQA Co). This effectively means that an “MQA validated” file can contain less than 16 bits of true musical data (since some is committed to lighting the fecking blue barf signal). What a fecking scam.

 

For the record: I have listened to a LOT of MQA from TIDAL. I have a fully decoding MQA DAC (see signature) and I can tell you this is a mixed bag at best.

[Rt66indierock]

1 hour ago, miguelito said:

Delicious... 

 

It is truly embarrasing that a 16 bit file will trigger the “blue dot of cash” (cash for MQA Co). This effectively means that an “MQA validated” file can contain less than 16 bits of true musical data (since some is committed to lighting the fecking blue barf signal). What a fecking scam.

 

For the record: I have listened to a LOT of MQA from TIDAL. I have a fully decoding MQA DAC (see signature) and I can tell you this is a mixed bag at best.

 

Love the "blue dot of cash" we're going to have to work on a logo.

[FredericV]

In a Dutch Facebook group managed by a Meridian dealer, the users actually believe that they are listening to 24/192 because their NAD amp is saying so.

https://www.facebook.com/photo.php?fbid=1834948226537180&set=gm.598310543850915&type=3&theater&ifg=1

It seems on bluesound, there's also 24/192 on the display of a NAD amp, and the user believes this blindly:

 

Basically they are listening to something like 17/96 (core decoder) upsampled to 24/192 (renderer).

[james45974]

2 hours ago, FredericV said:

Basically they are listening to something like 17/96 (core decoder) upsampled to 24/192 (renderer).

That is my big hangup (amongts others) about MQA is that you never know what you are getting.  This seems to be intentional, making people believe they are getting more than they really are.  I can upsample any number of ways here but I still know that for the majority of my recordings the source was a redbook rip, for instance.  DACS that use MQA  should display the original bit rate of the file, not the sampled number.  But then again, MQA doesn't seem to be about truth in advertising anyway! ;)

[FredericV]

Is MQA's compression a joke?

If we sabotage the 8 LSB bits in any way (blanking / random garbage / ascii string "MQASHILL" encoding), the spectrum changes after decoding:

 

In all four cases, the user sees 24/352.8 on his DAC display or screen / app of MQA enabled device, but does not know what he actually gets. The spectrum of the sabotaged files extends above 20 Khz, but Mans has shown this is due to aliasing.

Maybe the biggest joke is the fact that this little spectrum extention results in 50% larger PCM files (24 bits instead of 16 bits), and 100% larger flac files. Why? The 8 LSB bits in MQA don't compress well in FLAC.

First we took a new 2L.no MQA distribution file, and copied 20 secs to a wav file using sox:
 

sox  2L-053_04_stereo-DXD.mqa.flac 2L-053_04_stereo-DXD-20secs.mqa.wav trim 0 20

We stripped 8 bits using our tool, so we now have two 24/44.1 files with a duration of 20 secs, one where the 8 LSB bits have been blanked, and another which is a 20 secs copy of the real MQA file, which can both be decoded by MQA to 24/352.8.
 

$ du -h --si 2L-053_04_stereo-DXD.mqa-16in24bit.wav 2L-053_04_stereo-DXD-20secs.mqa.wav
5.3M    2L-053_04_stereo-DXD.mqa-16in24bit.wav
5.3M    2L-053_04_stereo-DXD-20secs.mqa.wav

Now if we encode both WAV files back to flac, let's observe the already impressive compression ratio difference:

 

$ flac -f --best 2L-053_04_stereo-DXD.mqa-16in24bit.wav 2L-053_04_stereo-DXD-20secs.mqa.wav

flac 1.3.0, Copyright (C) 2000-2009, 2011-2013  Josh Coalson & Xiph.Org Foundation
flac comes with ABSOLUTELY NO WARRANTY.  This is free software, and you are
welcome to redistribute it under certain conditions.  Type `flac' for details.

2L-053_04_stereo-DXD.mqa-16in24bit.wav: WARNING: legacy WAVE file has format type 1 but bits-per-sample=24
2L-053_04_stereo-DXD.mqa-16in24bit.wav: wrote 1633178 bytes, ratio=0.309
2L-053_04_stereo-DXD-20secs.mqa.wav: WARNING: skipping unknown chunk 'fact' (use --keep-foreign-metadata to keep)                                                    2L-053_04_stereo-DXD-20secs.mqa.wav: wrote 3394644 bytes, ratio=0.641

Now look at the file sizes:

$ du -h --si 2L-053_04_stereo-DXD.mqa-16in24bit.flac 2L-053_04_stereo-DXD-20secs.mqa.flac
1.7M    2L-053_04_stereo-DXD.mqa-16in24bit.flac
3.4M    2L-053_04_stereo-DXD-20secs.mqa.flac

So this means that only the redbook part of the MQA distribution file compresses with flac like normal music. The secret bits don't compress and therefore the filesize goes x2. While this is nothing new, the fact that it results in such a small difference in spectrum, makes it a waste of diskspace.

Most likely this is how MQA CD gets away with it. If someone can get me an MQA cd, we can do some measurements.

Furthermore, compressing the unfolded versions (both 24/88.2 wav) back to flac, shows a small difference is file size:
 

$ du -h --si 2L-053_04_stereo-DXD.mqa-16in24bit-unfolded.flac unfold-official-20secs.flac
5.7M    2L-053_04_stereo-DXD.mqa-16in24bit-unfolded.flac
6.0M    unfold-official-20secs.flac

 

This suggests a 5% reduction in entropy in the MQA decoder output, when the decoder did not have the 8 LSB bits to work with. But as the flac version of the input file for the decoder is 50% smaller for the 16 bits truncated version, this means MQA is not very efficient.

Also note that flac can't compress the output of the decoder back to the MQA flac distribution file sizes. This suggest garbage is added which does not compress well (garbage because of aliasing and higher noise floor).



 

[Don Hills]

I am surprised that the MQA decoder doesn't seem to complain when it is fed the invalid compressed data.