Blue or red pill?

[pkane2001]

2 hours ago, manisandher said:

 

I thought Paul's last analysis of the two analogue captures (done a while ago now) was very convincing that there were indeed measurable differences between the analogue captures:

 

 

If people still aren't convinced by my 9/10 ABX result and these measurements, I doubt anything will shift their entrenched beliefs.

 

Mani.

 

 

Mani, I did find some differences, but the analog captures were really poorly done. I don't know if it was a problem with the ADC or the connection, but there was a large amount of variation even between successive captures with no changes in software or hardware. 

 

I've been working on DeltaWave since then, and it now incorporates quite a few new analysis tools, including measuring and adjusting for non-linear phase differences, non-linear level differences, jitter, etc. 

 

By the way, since I added an ABX-style comparator to DeltaWave, I spent a lot of time testing it. Believe it or not, 9/10 is not that hard to get by just random guessing, or by selecting just A or just B repeatedly. I've seen this happen a number of times, more than I would've expected previously. It's unlikely, but it does happen. I've even had runs of up to about 14/15 just by random luck. That's why you need to repeat the test at least two or three more times. If you can get 9/10 on multiple runs, with not too many poor results in between, then you can be much more certain of the conclusion.

[sandyk]

1 hour ago, PeterSt said:

 

I have never said anything of the kind. Never.

Can you now, please, finally stop mixing up your ideas about audible differences in the exact same files, with differences I (and Mani for that sake) incur for by different playback means ?

Maybe write it on a wall somewhere so you won't forget ?

 

Kind regards,

Peter

I must have misunderstood you when you said this

Quote

Maybe you recall that in that thread of concern (blue/red pill) I warned a dozen times in advance about what you wanted to measure, couldn't

 

Different playback means can sound different, just as it does  with different types of S/W, even different S/W used to convert from .aiff to .wav . That's not unique. Most members wouldn't be too surprised about that.

Even playing from System Memory, or from 2 different storage locations can sound a little different.

 An SSD and a HDD for example. That was demonstrated years ago at a C.A. Symposium .

 IIRC, years ago Paul R demonstrated that a severely fragmented file can sound different too.

[manisandher]

5 hours ago, pkane2001 said:

By the way, since I added an ABX-style comparator to DeltaWave, I spent a lot of time testing it. Believe it or not, 9/10 is not that hard to get by just random guessing, or by selecting just A or just B repeatedly.

 

Sure. It's simple maths.

 

Anyone can try this. Toss a coin ten times. There is a 1 in 100 chance that you'll get heads (or tails) 9 times. Toss it enough times and you're very likely to get 9/10 at some point.

 

However, random guessing is not what I was doing in the blind test, hence why I scored 9/10 on my first attempt in the ABX.

 

[pkane2001]

Just now, manisandher said:

 

Sure. It's simple maths.

 

Anyone can try this. Toss a coin ten times. There is a 1 in 100 chance that you'll get heads (or tails) 9 times. Toss it enough times and you're very likely to get 9/10 at some point.

 

However, random guessing is not what I was doing in the blind test, hence why I scored 9/10 on my first attempt in the ABX.

 

 

Random or not, it is possible to get 9/10 result without trying. I've done it. It surprised me when I was just selecting A repeatedly that my score was so high. Caused me to spend a lot of time looking at the logic and I even re-implemented a more robust random number generator. And I still could duplicate this on occasion. I'm not saying you didn't hear a difference, there were differences in the analog captures. But a single 9/10 result is still not conclusive. Didn't you fail the first few attempts, or am I misremembering?

[rando]

Maybe he'll tip your driver or make some other token effort to get you down to chez Mansr to see the side of the coin he drew in all this.    

 

Cordially settling a matter is always better than arguing on the internet.  Still I have to ask you to refresh our memory of who did the ADC recording on that day?

[manisandher]

4 minutes ago, pkane2001 said:

Didn't you fail the first few attempts, or am I misremembering?

 

No, the first two attempts were just a random string, and I've explained at length that without a reference, it was impossible to identify A or B correctly. We conducted one set of 10 ABXs, and I scored 9/10.

 

If I bet you $100 against getting 9 heads (or tails) in 10 coin tosses, would you take it? Of course you wouldn't. No one in their right mind would. I didn't invite Mans up to my place to rely on luck in getting 9/10.

[pkane2001]

15 minutes ago, manisandher said:

 

No, the first two attempts were just a random string, and I've explained at length that without a reference, it was impossible to identify A or B correctly. We conducted one set of 10 ABXs, and I scored 9/10.

 

If I bet you $100 against getting 9 heads (or tails) in 10 coin tosses, would you take it? Of course you wouldn't. No one in their right mind would. I didn't invite Mans up to my place to rely on luck in getting 9/10.

 

And I wouldn't bet my house, my car or anything of value that 9/10 is not going to happen by chance. So here we are

 

[manisandher]

2 minutes ago, pkane2001 said:

And I wouldn't bet my house, my car or anything of value that 9/10 is not going to happen. So here we are

 

If the up side was the value of my house, my car, etc., I'd do it in a heart beat.

 

And I suspect you would too 

[Arpiben]

Hi Mani,

 

With no wish of interfering I just have a technical question.

As far as I understood DAC's analogue recordings were performed with DAC directly connected to Recorder. Fine.

A closer to reality  setting is to have DAC Output loaded with both amplifier and recording instrument during audio tests.

Was the above ever performed ?

Regards.

[mansr]

15 minutes ago, Arpiben said:

With no wish of interfering I just have a technical question.

As far as I understood DAC's analogue recordings were performed with DAC directly connected to Recorder. Fine.

A closer to reality  setting is to have DAC Output loaded with both amplifier and recording instrument during audio tests.

Was the above ever performed ?

I wanted to do it that way, but Mani refused.

[alfe]

1 hour ago, manisandher said:

 

 

 

I scored 9/10 on my first attempt in the ABX.

 

Still below 10/10 😂

[manisandher]

28 minutes ago, Arpiben said:

Hi Mani,

 

With no wish of interfering I just have a technical question.

As far as I understood DAC's analogue recordings were performed with DAC directly connected to Recorder. Fine.

A closer to reality  setting is to have DAC Output loaded with both amplifier and recording instrument during audio tests.

Was the above ever performed ?

Regards.

 

No problem...

 

I think it's useful to split this into 2:

 

1. blind ABX test

 

I wanted the purest path possible during the ABX, so no preamps or splitters. The priority was to ensure the output of the DAC was bit-identical in both cases, A and B. And it was.

 

2. Analysis

 

Only if the ABX result looked interesting would we bother to capture and analyse the analogue output of the DAC. I scored 9/10 in the ABX (1% probability by guessing alone), so we went ahead and took the analogue captures.

 

Even if we did not take any analogue captures, the result of the ABX would have stood.

 

Mani.

[manisandher]

5 minutes ago, alfe said:

Still below 10/10 😂

 

Yep, I still can't explain how I managed to get one wrong .

 

Mani.

[sandyk]

2 hours ago, alfe said:

Still below 10/10 😂

 Al

I would be willing to bet that once Mani's ears " locked in" to the differences that he would have been able to repeat that several more times before needing a refreshment break. On most occasions such as this, the setting up times reduce the time available.

 

Alex

[sandyk]

3 hours ago, mansr said:

I wanted to do it that way, but Mani refused.

 

And I don't blame him. He would be trying to maximise the audible differences so that they would hopefully even be heard by you,.

 

[fas42]

The real silliness in all this is the belief that even a 100% properly done ABX actually 'proves' anything about what's going on with human hearing - there is a great conceit with much of science in the belief that they can always get a handle on how the human organism operates using simple laboratory methods, and there have been some terrible disasters as a result of this type of thinking.

 

I would be quite certain that you could have two people in a room participating in the identical ABX, one who has strong conviction that they can hear the differences; and the other believing it is all nonsense; the former would get excellent results, the latter would achieve almost perfect random scoring - because the need to be right would override anything his senses were telling him ... and absolutely nothing would be gained, yet again.

[manisandher]

23 hours ago, rando said:

Still I have to ask you to refresh our memory of who did the ADC recording on that day?

 

The original analogue captures (DAC's analogue output -> Tascam DA-3000 -> 24/176.4 files) were undertaken by Mans and myself immediately after the ABX. I took some further captures after Mans left to help in analysis.

 

The Tascam DA-3000 has pretty decent specs and is a well-considered ADC, but seems lacking for the job in hand. The differences in audibility between A and B during the ABX were not down to any obvious differences in frequency response.

 

Mani.

[Jud]

On 8/6/2019 at 11:39 AM, manisandher said:

 

The original analogue captures (DAC's analogue output -> Tascam DA-3000 -> 24/176.4 files) were undertaken by Mans and myself immediately after the ABX. I took some further captures after Mans left to help in analysis.

 

The Tascam DA-3000 has pretty decent specs and is a well-considered ADC, but seems lacking for the job in hand. The differences in audibility between A and B during the ABX were not down to any obvious differences in frequency response.

 

Mani.

 

I recall some offhand remarks in the forum to the effect that the measurement equipment ought to have specs in the range of 10x as good as the device being measured.  Wonder if this means that unless you have one of the world's premiere ADCs measuring a good DAC, you're essentially measuring the limits of the ADC.

[manisandher]

1 hour ago, Jud said:

Wonder if this means that unless you have one of the world's premiere ADCs measuring a good DAC, you're essentially measuring the limits of the ADC.

 

That's probably the case.

 

But it's important to point out that the ADC was only being used to analyse what might be going on. It had absolutely no influence on the outcome of the ABX itself.

 

Mani.

[fas42]

Even a relatively poor measuring ADC would normally have the ability to register the differences in the output - which is all that is needed. It's not absolute performance that's being measured - but rather, a detectable variation.

 

Part of the Art of Science is to craft a method of observing something, when the measurement tools are "not good enough!" - not much use throwing up one's arms, and saying, "Ye canna do it!!"

[manisandher]

14 minutes ago, pkane2001 said:

Curious about the recording chain. Was there something that was configured in XXHighEnd or was there something else in the digital path between the original file and the digital recorder that caused the recording to be so different in an all-digital chain?

 

 

 

XXHighEnd was performing a couple of operations on the original file before passing it through to the DAC:

 

1. upsampling from 44.1k to 176.4k (DAC being used was non-oversampling)

2. attenuating (DAC was connected directly to power amp)

 

I attached the original file in case you wanted to take a look at it. But what we're really interested in, of course, are any differences between the 30 digital captures.

 

Edit: That's why I've always been careful to use the term bit-identical, as opposed to bit-perfect when referring to the digital captures.

[pkane2001]

9 minutes ago, manisandher said:

 

XXHighEnd was performing a couple of operations on the original file before passing it through to the DAC:

 

1. upsampling from 44.1k to 176.4k (DAC being used was non-oversampling)

2. attenuating (DAC was connected directly to power amp)

 

I attached the original file in case you wanted to take a look at it. But what we're really interested in, of course, are any differences between the 30 digital captures.

 

Edit: That's why I've always been careful to use the term bit-identical, as opposed to bit-perfect when referring to the digital captures.

 

That's still strange, as the changes to the original digital file are significant and certainly not even close to being bit-identical between the file and the recording. Attenuation would not account for this, since DeltaWave compensates for level differences.

[pkane2001]

18 minutes ago, manisandher said:

 

XXHighEnd was performing a couple of operations on the original file before passing it through to the DAC:

 

1. upsampling from 44.1k to 176.4k (DAC being used was non-oversampling)

2. attenuating (DAC was connected directly to power amp)

 

I attached the original file in case you wanted to take a look at it. But what we're really interested in, of course, are any differences between the 30 digital captures.

 

Edit: That's why I've always been careful to use the term bit-identical, as opposed to bit-perfect when referring to the digital captures.

 

There are some differences at the beginning of the recording, as we discussed, but bit-perfect comparison between the A and B files after that. For example:

 

 

The result says that the files samples match to 99.86%. For a 14 second recording at 176.4k this is about 3500 samples or about 20ms difference at the start.

[manisandher]

31 minutes ago, pkane2001 said:

That's still strange, as the changes to the original digital file are significant and certainly not even close to being bit-identical between the file and the recording. Attenuation would not account for this, since DeltaWave compensates for level differences.

 

It'll [edit: the HF attenuation will] be down to XXHighEnd's Arc Prediction filtering. Not a problem in the test though - both A and B were going through the exact same processing.

[manisandher]

13 minutes ago, pkane2001 said:

The result says that the files samples match to 99.86%. For a 14 second recording at 176.4k this is about 3500 samples or about 20ms difference at the start.

 

I can assure you (and everyone else) there was no 'tell' within the first 20ms of each music sample! And in any event, the mismatch was just due to the digital recorder's auto-start function. As you might imagine, I no longer use the auto-start function.

 

The signal reaching the DAC remained bit-identical throughout the ABX.