Archimago‘s 16/44 test

[mcgillroy]

...first infos up on his site. No stats yet but he unblinded the devices. Seems like - uhhhh uhm - I really liked the iPhones sound: 

 

http://archimago.blogspot.com/2019/05/blind-test-results-part-1-do-digital.html

[esldude]

Looks like he made similar choices as I did when I posted a similar set of tracks a few years ago.  I used a laptop computer, android phone, and a couple good DACs.  I'm sure he'll get all the same complaints.  Will be interesting to see his results since he has 101 responses.  

 

You say you dig the iPhone 6 sound? Maybe you are in good company.  John Swenson preferred the Android phone sound in my little test.  

[fas42]

Simplicity wins. I wouldn't be the least bit surprised if the iPhone output comes out on top both subjectively, and in an objective testing of the quality - the local audio friend has been using 'cheap' media player equivalents for years now, and has achieved outstanding results at times.

 

Many people have a psychological need for things to be Big, and/or Expensive - before they can be considered, Good ... if one can overcome that hurdle, then it's possible to achieve excellent value for money in the audio game ... .

[fas42]

Umm, I was a slack bastard - I never actually went ahead and did the poll - but reacted strongly to the items at the time when it was posted ... found where I had left the uncompressed samples of the Crowd Chant track, and had a listen, again. The steady drum kick through the track is a good sound element to follow, has a hypnotic steadiness; allows a sense of the quality to build - and makes it obvious how the playback varies between the devices.

[Paul R]

Most irritating, I lost where I saved my choices.  I will probably never know which one I chose blind now.

 

-Paul 

 

[fas42]

Ahh, those senior moments ...

[mcgillroy]

Looking at my notes (messy) I had B as favourite, followed by C and D. Didn't like A. But frankly it was close, surprisingly close. 

 

Seems like Apple has their shit together. Perhaps a world-class audio team can do something right.

[sandyk]

37 minutes ago, mcgillroy said:

Looking at my notes (messy) I had B as favourite, followed by C and D. Didn't like A. But frankly it was close, surprisingly close. 

 

 It could also be possible that no device is a clear winner with all files, given the different musical choices.

 I couldn't be bothered voting, and in fact I just had a quick listen a few minutes ago , with nothing in C or D impressing me.

 However, this time of night my BP is all over the place, and I mightn't choose the same say 1st thing in the morning .

 TBH, I thought they ALL were varying degrees of mediocrity !

[fas42]

Why do the outputs from different digital devices sound different? Well, they are different - obviously so ...

 

These are the waveforms of the samples posted by Archimago, of the "Crowd Chant" track - for in blue, the iPhone, and red, the Oppo - aligned, showing a point 3.13 secs into the track ... do you think you would hear a difference?

 

[Sonicularity]

Part II is up.

 

http://archimago.blogspot.com/2019/05/blind-test-results-part-2-do-digital.html

[fas42]

Sorta hilarious seeing those "brilliant" performance figures put up - yet when one looks at the actual waveforms there are monstrous variations, in real world captures of the outputs, as above - so, "something is lying" ... but what?

[Rexp]

I thought a and c were equal best followed by d then b, doesn't seem to correlate wirh the measurements-no surprise. 

[shtf]

On 5/4/2019 at 11:14 AM, mcgillroy said:

...first infos up on his site. No stats yet but he unblinded the devices. Seems like - uhhhh uhm - I really liked the iPhones sound: 

 

http://archimago.blogspot.com/2019/05/blind-test-results-part-1-do-digital.html

 

I like Archimago and I appreciate some of his work especially his demeanor and seemingly due diligence.  But with all due respect I find neither this test nor his 24-bit vs 16-bit test objective.

 

In fact, based on Archimago's findings, the only thing I can find overwhelmingly conclusive or reaffirming is that high-end audio's weakest link by far is the majority's ability to discern / interpret what we hear.  And that includes musicians and engineers as well as enthusiasts, manufacturers and designers, dealers, distributors, reviewers, and editors.  But this is old news.

 

And to the contrary, even if well-intentioned I can't help but think these subjective tests as little more than potentially damaging to high-end audio as well as potentially self-serving.  Even though I do not think that's the intention.

 

And this not to say I possess well-train ears (though I hope to possess well-enough trained ears to get by) nor is this to say I find significant differences between 24-bit and 16-bit but for one reason or antoher there are often times small but distinct differences.  And some of those differences may have little / nothing to do with the format itself.

 

[Ralf11]

What do you mean by "objective"?

[shtf]

34 minutes ago, Ralf11 said:

What do you mean by "objective"?

 

Did you even try to take the meaning of my statement on its face?

[Ralf11]

Did you even try to think about the multiple meanings of the term ????

[shtf]

17 minutes ago, Ralf11 said:

Did you even try to think about the multiple meanings of the term ????

 

You're right.  There are many potential meanings and interpretations of the use of the word objective.  

 

So why do you bother trying to lead me down this trite rabbit hole?

[esldude]

3 hours ago, shtf said:

 

I like Archimago and I appreciate some of his work especially his demeanor and seemingly due diligence.  But with all due respect I find neither this test nor his 24-bit vs 16-bit test objective.

 

In fact, based on Archimago's findings, the only thing I can find overwhelmingly conclusive or reaffirming is that high-end audio's weakest link by far is the majority's ability to discern / interpret what we hear.  And that includes musicians and engineers as well as enthusiasts, manufacturers and designers, dealers, distributors, reviewers, and editors.  But this is old news.

 

And to the contrary, even if well-intentioned I can't help but think these subjective tests as little more than potentially damaging to high-end audio as well as potentially self-serving.  Even though I do not think that's the intention.

 

And this not to say I possess well-train ears (though I hope to possess well-enough trained ears to get by) nor is this to say I find significant differences between 24-bit and 16-bit but for one reason or antoher there are often times small but distinct differences.  And some of those differences may have little / nothing to do with the format itself.

 

Why do you think this is damaging to high end?  Because high enders have to maintain the idea their hearing abilities are utmost instead of the truth, which is their hearing isn't that good and easily fooled?  Or some other reason.

[Ralf11]

This is exactly the sort of approach that is needed.

 

I only regret he didn't compare $20,000 and $2,000 DACs.

[shtf]

2 hours ago, esldude said:

Why do you think this is damaging to high end?  Because high enders have to maintain the idea their hearing abilities are utmost instead of the truth, which is their hearing isn't that good and easily fooled?  Or some other reason.

 

Good question.  Though in and of itself your own question is potentially damaging / unhealthy.

 

Let’s assume for the sake of argument that a clear majority of high-end audio participants of every kind believe we inherit at birth the ability to discern / interpret what we hear and have no need to nurture that ability so long as we passed a hearing test two years ago.  Now in the visual world of fine art, a genuine connoisseur of fine art would be laughing his ass off if somebody told him that we all inherit at birth the ability to discern / interpret fine art and have no need to nurture that ability.

 

Nevertheless, since it seems the vast majority of listeners lack such listening skills, any findings by polling the same using a series of questionable / subjective tests only further substantiate the fallacy that we all inherit at birth this ability to discern / interpret what we hear and that our equipment really doesn’t matter.

 

First we test those potentially lacking the ability to discern / interpret what they hear by providing them 4 potentially base “digital players”, actually at least one is a digital processor only.  And when the outcome as expected is that the bulk of participants cannot audibly discern potential differences between digital processors then the logical conclusion is that digital processors all (not just 4 low-end processors) sound far more similar than different.

 

Then we conduct a similar test using similar participants to discern differences between 24-bit and 16-bit digital.  Low and behold we have similar findings.

 

But why stop there?  Always have to look at an endeavors potential outcome as well as its actual outcome, right?  Following this pattern, it only makes sense to then conduct a similar test for preamps, then cables, then all other parts of the audio vineyard and achieve similar findings for all parts of the vineyard.

 

I’m already aware of some so-called “Audio Experts” who’ve proclaimed for years that all cables sound identical and all components sound identical because ALL retain the fidelity of the input signal.

 

Ultimately, how do Archimago’s questionable findings from his questionable tests help keep us from thinking everything sounds identical like the Audio Expert I mentioned above?   If there was any substance to any of this, then there’s no more need to improve anything and hence, we have no more need for R&D, no need for new technologies, strategies, methods, etc. because it all sounds pretty much the same.  So the industry might as well focus on the bling-bling because ain’t nobody gonna’ buy anything for performance differences because tests like these only confirm that everything sounds pretty much identical.  (BTW, we're already there).

 

In reality, these types of tests and findings only demonstrate that the vast majority of industry participants are playing T-ball with their hobby and they are seeking to drag everybody else down to their lowest common denominator.  When the better alternative should be to get the T-ballers to strive to better themselves by demonstrating that a national league exists and hopefully T-ballers at least start dreaming they too could one day play in the national league and that there is opportunity for personal growth and increased pleasure. 

 

When those who lack the ability to discern / interpret what they hear should be focused on training themselves or spending time and resources to have somebody else train them so they are able to become better trained and strive for improved levels of musicality that greatly enhances the potential enjoyment that high-end audio can provide.

 

Rather, the only real outcome of those who refuse to train or hone their listening skill along with such tests and findings as these are that with each passing year a greater percentage of enthusiasts become more comfortable and confident with their inability to discern / interpret what they hear and the more they mock those that have some ability to discern / interpret what they hear until eventually they stop participating in audio forums because it’s no longer worth the friction.

 

How is any of this potentially damaging / unhealthy for high-end audio, you ask?  Because ultimately from a performance perspective this base-level mindset keeps the entire industry far closer to its infancy than it rightfully should be.  If the pedal-to-the-metal, balls-to-the-walls drag racing industry maintained this same mindset all these years, I’d guess we’d still be arriving at the drag strip on Saturday nights with the family station wagon ready to race while those who removed their hubcaps to lessen wind resistance would be mocked.

 

[fas42]

Archimago has fallen down the usual rabbit hole that "measurements are everything!" - of course in the real world everyone knows this is nonsense, that's why you have car magazines where people who are allowed to push vehicles to their limits recount what the actual experience inside the beasts is like; they quickly make up a list of weaknesses and shortcomings, which no set of spec's or instrumented readings has a chance of providing.

[kumakuma]

8 minutes ago, fas42 said:

Archimago has fallen down the usual rabbit hole that "measurements are everything!" - of course in the real world everyone knows this is nonsense, that's why you have car magazines where people who are allowed to push vehicles to their limits recount what the actual experience inside the beasts is like; they quickly make up a list of weaknesses and shortcomings, which no set of spec's or instrumented readings has a chance of providing.

 

Deleted

[shtf]

3 hours ago, esldude said:

Why do you think this is damaging to high end?  Because high enders have to maintain the idea their hearing abilities are utmost instead of the truth, which is their hearing isn't that good and easily fooled?  Or some other reason.

 

I neglected to mention some of the ancillary damages of an industry kept near its infancy from a performance perspective.

 

1.  There are those dealers and distributors who realize the majority possessing this base level state of listening skills and they capitalize on it in audio forums with their bling only, playing tag team with one another about the "performance" of products and making money via their infomercials masquarading as threads in the forums when the gullible are just gobbling it up and the professionals are making money from it and developing a cult of followers.

 

2.  It becomes far easier for charlatans like Bob Stuart of MQA, LTD to introduce a product like MQA with just a few key endorsements to hoodwink the majority into thinking a supposedly new "hi-rez" format can now make cows jump over the moon.  It also becomces far easier for supposedly industry leaders like Atkinson and Harley to "guide" the industry toward a technology like a particular hi-rez format, a certain multi-channel config, etc.

 

3.  It becomes far easier for the science-minded, engineering-minded, and psuedo-science-minded types to become paper tigers of audio forums that mislead the masses into thinking they are the real audio experts and that they have the answers.  When in reality they are potentially more lost than the majority when it comes to real high-end audio performance because they've convinced themselves and many of their followers that listening skills no longer matter and performance we can SEE is all that matters.  In an audio-only industry, mind you.

 

 

[sandyk]

1 hour ago, shtf said:

When those who lack the ability to discern / interpret what they hear should be focused on training themselves or spending time and resources to have somebody else train them so they are able to become better trained and strive for improved levels of musicality that greatly enhances the potential enjoyment that high-end audio can provide.

 

That is why it is helpful ,and educational, to be part of a group that regularly meets at different houses (if possible) to compare and listen to different types of electronics from various price ranges , including speakers, preferably under NON sighted conditions.

 Measurements aren't the Be-all, End-all guide as to quality, UNLESS there are  quite obvious  differences in the output compared with the original input.

[esldude]

37 minutes ago, shtf said:

 

Good question.  Though in and of itself your own question is potentially damaging / unhealthy.

 

Let’s assume for the sake of argument that a clear majority of high-end audio participants of every kind believe we inherit at birth the ability to discern / interpret what we hear and have no need to nurture that ability so long as we passed a hearing test two years ago.  Now in the visual world of fine art, a genuine connoisseur of fine art would be laughing his ass off if somebody told him that we all inherit at birth the ability to discern / interpret fine art and have no need to nurture that ability.

 

Nevertheless, since it seems the vast majority of listeners lack such listening skills, any findings by polling the same using a series of questionable / subjective tests only further substantiate the fallacy that we all inherit at birth this ability to discern / interpret what we hear and that our equipment really doesn’t matter.

 

First we test those potentially lacking the ability to discern / interpret what they hear by providing them 4 potentially base “digital players”, actually at least one is a digital processor only.  And when the outcome as expected is that the bulk of participants cannot audibly discern potential differences between digital processors then the logical conclusion is that digital processors all (not just 4 low-end processors) sound far more similar than different.

 

Then we conduct a similar test using similar participants to discern differences between 24-bit and 16-bit digital.  Low and behold we have similar findings.

 

But why stop there?  Always have to look at an endeavors potential outcome as well as its actual outcome, right?  Following this pattern, it only makes sense to then conduct a similar test for preamps, then cables, then all other parts of the audio vineyard and achieve similar findings for all parts of the vineyard.

 

I’m already aware of some so-called “Audio Experts” who’ve proclaimed for years that all cables sound identical and all components sound identical because ALL retain the fidelity of the input signal.

 

Ultimately, how do Archimago’s questionable findings from his questionable tests help keep us from thinking everything sounds identical like the Audio Expert I mentioned above?   If there was any substance to any of this, then there’s no more need to improve anything and hence, we have no more need for R&D, no need for new technologies, strategies, methods, etc. because it all sounds pretty much the same.  So the industry might as well focus on the bling-bling because ain’t nobody gonna’ buy anything for performance differences because tests like these only confirm that everything sounds pretty much identical.  (BTW, we're already there).

 

In reality, these types of tests and findings only demonstrate that the vast majority of industry participants are playing T-ball with their hobby and they are seeking to drag everybody else down to their lowest common denominator.  When the better alternative should be to get the T-ballers to strive to better themselves by demonstrating that a national league exists and hopefully T-ballers at least start dreaming they too could one day play in the national league and that there is opportunity for personal growth and increased pleasure. 

 

When those who lack the ability to discern / interpret what they hear should be focused on training themselves or spending time and resources to have somebody else train them so they are able to become better trained and strive for improved levels of musicality that greatly enhances the potential enjoyment that high-end audio can provide.

 

Rather, the only real outcome of those who refuse to train or hone their listening skill along with such tests and findings as these are that with each passing year a greater percentage of enthusiasts become more comfortable and confident with their inability to discern / interpret what they hear and the more they mock those that have some ability to discern / interpret what they hear until eventually they stop participating in audio forums because it’s no longer worth the friction.

 

How is any of this potentially damaging / unhealthy for high-end audio, you ask?  Because ultimately from a performance perspective this base-level mindset keeps the entire industry far closer to its infancy than it rightfully should be.  If the pedal-to-the-metal, balls-to-the-walls drag racing industry maintained this same mindset all these years, I’d guess we’d still be arriving at the drag strip on Saturday nights with the family station wagon ready to race while those who removed their hubcaps to lessen wind resistance would be mocked.

 

So in essence, you think there are a small number of people who are much more discerning of sound quality differences.  And if we think no improvements are possible, none will be forthcoming as no one will look for them.  Plus that many/most audiophiles could learn to discern more than they currently do.  Which would provide a better market for improved gear rather than designing for the lowest common aural acuity.  You want an arms race of performance with various companies pushing each other to higher and higher levels of good results.  I think I've not misrepresented you here. 

 

So laying aside Archimago's test for the moment, my question for you would be do you think there is no end to how well humans can hear? 

 

If you say you think there is not, then your reasoning makes some sense though just the nature of things means as results become more rarefied it will apply to fewer and few people.  

 

If you think there is a limit, then what do you think should occur if we can make gear known to be faultless and perceptually perfect?

Laying aside whether or not we are there yet.  (my opinion for what is worth is some of our gear does this and some does not). 

 

I think there are some faults in other parts of your post.  One being the assumption no differences were heard.  We don't know the results of that yet. 

 

I’m already aware of some so-called “Audio Experts” who’ve proclaimed for years that all cables sound identical and all components sound identical because ALL retain the fidelity of the input signal.

 

The above is a straw man.  

 

I don't know exactly what Archimago had in mind.  I think it was more of an interesting test for people to see if they could hear a difference they thought would be easy.  I think if you asked audiophiles if they could hear the difference in an iPhone and a really good DAC they'd say yes.  This is where you might say well trained listeners could, but most audiophiles will not.  So the results would mislead us if no statistical difference of significance is found.  So following your thinking you believe this is negative.  I'd say, "well if these people can't hear the difference why shouldn't they learn it is true?" I don't think Archimago would claim his test should be used as a basis for determining ultimate performance levels needed for the entire industry.   It might say it was good to know most people can't hear the difference or the difference is much smaller than they imagined.  And again I don't know what the results will be. 

 

I've posted files of various kinds and that was my thinking.  Interesting experience, but given the various listener abilities, and gear, and how much effort a listener may have invested it is not conclusive.  The most recent one with 8th generation copies, is interesting in the sense you are thinking about it. The results didn't indicate people in general could hear a difference.  I could ABX those files.  In time some other people did.  We have to remember a null result doesn't prove no difference.  It is when we get positive results that things are interesting.  In time I determined what it was that let me hear the 8th generation copies vs originals.  And it told me some people gave up or did not know how to listen carefully for a difference.  That will be a problem with any public test that doesn't vet the gear, conditions, or listeners.