Misleading Measurements

[pkane2001]

1 hour ago, bluesman said:

I never said anything about how well or correctly it functions. Do you not understand or believe that an undetected secondary effect of the app in question might affect its use and cause misleading results?  
 

For example, imagine using the app to add an anomaly we’ll call anomaly X to a music file to see if you can hear it.  But unknown to you and the creator of the app, it also compresses the output at one end of the spectrum enough to make an audible difference to you.  
 

You hear a difference in SQ between app off and app on, so you reasonably conclude that you can hear the effect of anomaly X on the source file.  What you’re actually hearing is the effect of slight compression, but you didn’t know it was added and neither did the app designer because it was never considered and not looked for during the validation process.

 

If you don’t understand this valid real world example and see that it’s a practical use cases for us as audiophiles, I don’t know what else I can add to help you.

The part I don’t understand is the demand to provide some sort of absolute validity testing.
 

The problem is that such proof cannot be provided, simply because no matter how much testing is done, there will always be something that someone will suggest as another test that wasn’t performed yet, and therefore the software is not valid until more tests are done. 

 

I’ve done my own testing to verify to my satisfaction that the app does work. Others done this as well. Look at the ASR thread. You can do your own testing to your heart’s content, or ask someone else to do it if you don’t know how. I might be able to provide test results if a specific test is suggested that I’ve already performed. 
 

This is why I’ve been insisting on using a specific  validity test rather than some nebulous, general “validation”, which is ill defined and cannot be achieved.

[SoundAndMotion]

@bluesman I realize this is frustrating for you, but I think we're about to punch through a communication barrier.

 

What is missing in your example is that anomaly X and the compression are both manifestations of the same thing: a particular nonlinear time-domain transfer function. If you want to add anomaly X, you have to be aware that the compression comes with it.

If you get a dog, fully aware that you'll have to feed it, don't be surprised that you also have to clean up the poo. It's part of the same process. If you didn't know, don't blame the kennel or the dog food company.

I'm sure Paul knows that creating a compressor and getting harmonic distortion go together.

I disagree that it is a reasonable conclusion that hearing a difference means it is due to anomaly X. You can't expect to create anomaly X without knowing that your method also added compression. They are, in this case, inextricably linked.

Paul has created a nonlinear time-domain transfer function generator. He lets you set it up by specifying the harmonic content. No one can offer you an anomaly X generator that doesn't also add compression, if anomaly X is addition of certain harmonic distortions.

 

I think I understand what you are saying. I hope you're not so frustrated that you won't try to understand what I'm saying.

[fas42]

I doff my hat to you gents ... I don't have the stamina anymore to indulge in intense wrestling matches like this - interesting to watch from a distance; eventually it will stop, with absolutely zero gained by anyone - and such has been the way of audio for decades now, of course ... 🙂.

[Audiophile Neuroscience]

51 minutes ago, fas42 said:

I doff my hat to you gents ... I don't have the stamina anymore to indulge in intense wrestling matches like this - interesting to watch from a distance; eventually it will stop, with absolutely zero gained by anyone - and such has been the way of audio for decades now, of course ... 🙂.

 

Thanks Frank, my irony meter just exploded !!

[kumakuma]

14 minutes ago, Audiophile Neuroscience said:

 

Thanks Frank, my irony meter just exploded !!

 

As did mine.

 

I can't help but feel that if he stopped spamming the sh*t out of this site with repetitive posts, he might have the energy to actually contribute in a meaningful way. 

[fas42]

The thing is, there are different types of of back and forth - when people throw in lots of technical brouhaha which has zero meaning for the discussion, that's when my head explodes ... 😉.

[fas42]

4 minutes ago, kumakuma said:

 

As did mine.

 

I can't help but feel that he stopped spamming the sh*t out of this site with repetitive posts, he might have the energy to actually contribute in a meaningful way. 

 

You mean, in a way that fits in with your world view, of course ... 😉.

[kumakuma]

Just now, fas42 said:

 

You mean, in a way that fits in with your world view, of course ... 😉.

 

No, I don't. 

[Audiophile Neuroscience]

Well, I will have to just survive the veiled slurs on my motivations and character #776 #780 #781 but that aside nothing here or in the other dedicated thread changes one inconvenient truth.

 

I see no evidence that the app in question has  "been validated through independent testing" as claimed. That is not to say it doesn't do as claimed.There is talk of self-validation and doing your own measurements and theory/fact about how Transfer functions work that produce a reliable model. I am open to this possibility.Trust but verify independently

 

The new development appears to be that the "independent testing" confirming validity is an Audio "Science" Review internet forum thread and additionally a blog site conducted a survey using the app. For now, let's just say this might be seen as questionable verification.YMMV

[bluesman]

And we just returned to “Go!” without collecting $200.  It started here:
 

On 7/11/2020 at 1:48 PM, bluesman said:

I’m trying to understand if and how digital tones intermodulate with each other, not how IM distortion products are created.  There’s a big difference between the two.

 

Then we got this:

On 7/11/2020 at 3:57 PM, pkane2001 said:

Any frequencies that are not in the original signal are distortion. IMD happens in the analog domain, except for some special software or plugins  designed to simulate it, like my DISTORT.

Within the subsequent drivel was this:

On 7/11/2020 at 9:37 PM, pkane2001 said:

I suspect that if it was possible to completely remove IM from, say, a guitar or piano or human voice, it would sound completely unnatural to us.

This is at the heart of my belief and suggestion that intermodulation among instrumental notes is essential- and it is recorded as part of a musical performance.

 

But then came.....

On 7/15/2020 at 10:25 PM, pkane2001 said:

DISTORT applies a non-linear transfer function to any desired signal. This is a simple mathematical operation. The result is measured using a spectral plot. You can confirm that it generates correct IMD frequencies with two or more tones, as these are easily verifiable by comparing the result with any text on IMD by specifying the same frequencies/amplitudes in DISTORT, or by calculating various sums and differences yourself.

This strongly suggests to me and to users of the app that DISTORT can create IMD and only IMD by applying an unspecified nonlinear transfer function to any signal.  Because my knowledge and experience suggest that there would have to be other effects besides IMD, I asked how we knew that generating IMD is the only thing the app does when set to this function.  To me, this is part of validation - does it have any “side effects”?
 

The responses totally ignored this very valid concern until this popped up today:

3 hours ago, SoundAndMotion said:

What is missing in your example is that anomaly X and the compression are both manifestations of the same thing: a particular nonlinear time-domain transfer function. If you want to add anomaly X, you have to be aware that the compression comes with it.

And this is exactly where I was hoping to prompt someone to go, so I wouldn’t come off as argumentative.  Of course there are other effects besides the desired one when adding a nonlinear stage to a signal path - and not telling users this lets them believe that the effect they’ve dialed in is the only effect they’re hearing.  I don’t quite understand how SoundandMotion suddenly became a spokesperson here, but the nail has now been hit on the head.  A measurement without context is as misleading if you’re relying on it in a source signal as it is when evaluating playback equipment.

 

Would any of you want a vaccine that was validated as effective against an infectious agent but was not validated to have a lack of deleterious autoimmune side effects? I hope not.

[Audiophile Neuroscience]

back partially on topic:

 

The whole 'self- validation' issue still strikes me as a user satisfaction method based on the belief that the simulation model is correct (which may be correct) and is satisfied with the outcome. Each end user "verification" of "objective data" adds to the anecdotal pool of potentially misleading measurements.

 

It seems to me that people like @Superdad cop a lot of flack from objectivists for using a self-satisfaction criterion based on listening. The end user hears a difference (which may be correct) and each end user "verification" adds to the anecdotal pool but they get admonished for not producing some kind of independent validation.

 

I see possible misleading measurements and a double standard emerging. If its good for the goose it's good for the gander.

[Audiophile Neuroscience]

15 minutes ago, bluesman said:

Would any of you want a vaccine that was validated as effective against an infectious agent but was not validated to have a lack of deleterious autoimmune side effects? I hope not.

 

I would also want a vaccine that doesn't "self-validate" for efficacy, even if I could do my own IgG immunofluorescent antibody assays and feel I could interpret the results as inferring immunity. Call me crazy but I want independent testing before I get jabbed.

[kumakuma]

25 minutes ago, Audiophile Neuroscience said:

It seems to me that people like @Superdad cop a lot of flack from objectivists for using a self-satisfaction criterion based on listening. The end user hears a difference (which may be correct) and each end user "verification" adds to the anecdotal pool but they get admonished for not producing some kind of independent validation.

 

I see possible misleading measurements and a double standard emerging. If its good for the goose it's good for the gander.

 

The question I have is why aren't you pestering @Superdad for his independent validation data in the same way you are pestering Paul, especially considering that @Superdad is selling commercial products while Paul is charging nothing...

 

Seems like you're the one with the double standard.  👺

[pkane2001]

6 minutes ago, bluesman said:

And we just returned to “Go!” without collecting $200.  It started here:
 

 

Then we got this:

Within the subsequent drivel was this:

This is at the heart of my belief and suggestion that intermodulation among instrumental notes is essential- and it is recorded as part of a musical performance.

 

But then came.....

This strongly suggests to me and to users of the app that DISTORT can create IMD and only IMD by applying an unspecified nonlinear transfer function to any signal.  Because my knowledge and experience suggest that there would have to be other effects besides IMD, I asked how we knew that generating IMD is the only thing the app does when set to this function.  To me, this is part of validation - does it have any “side effects”?
 

The responses totally ignored this very valid concern until this popped up today:

And this is exactly where I was hoping to prompt someone to go, so I wouldn’t come off as argumentative.  Of course there are other effects besides the desired one when adding a nonlinear stage to a signal path - and not telling users this lets them believe that the effect they’ve dialed in is the only effect they’re hearing.  I don’t quite understand how SoundandMotion suddenly became a spokesperson here, but the nail has now been hit on the head.  A measurement without context is as misleading if you’re relying on it in a source signal as it is when evaluating playback equipment.

 

Would any of you want a vaccine that was validated as effective against an infectious agent but was not validated to have a lack of deleterious autoimmune side effects? I hope not.

 

@bluesman you are missing some basic etiquette in communicating, and your ability to clearly explain your thoughts is sorely lacking.

 

I've no clue what other effects you are talking about, since only the non-linear stage is what is being claimed and applied in the simulation. Whether you understand it or not, there are no other effects or context to describe. None. y = f(x). As simple as that.

 

What do you mean by "DISTORT can create IMD and only IMD"? There's no such thing, and nowhere did I claim this. What I said, at least a dozen times, is that DISTORT will construct a nonlinearity and that the same non-linearity will generate HD when applied to a single tone, and IMD when applied to multiple tones. Any other conclusions again, are all in your head.

 

I see that I was right before that you misinterpret and misread things, and then become nasty in communications based on your misunderstanding. That's about my limit. Have a great life. 

 

 

[Audiophile Neuroscience]

9 minutes ago, kumakuma said:

 

The question I have is why aren't you pestering @Superdad for his independent validation data in the same way you are pestering Paul, especially considering that @Superdad is selling commercial products while Paul is charging nothing...

 

Seems like you're the one with the double standard.  👺

 

AFAIK @Superdad never said he had "independent validation" but they are nonetheless searching. The only double standard emerges when independent validation is used as a weapon in one circumstance and considered unnecessary in another.

[pkane2001]

1 hour ago, Audiophile Neuroscience said:

back partially on topic:

 

The whole 'self- validation' issue still strikes me as a user satisfaction method based on the belief that the simulation model is correct (which may be correct) and is satisfied with the outcome. Each end user "verification" of "objective data" adds to the anecdotal pool of potentially misleading measurements.

 

It seems to me that people like @Superdad cop a lot of flack from objectivists for using a self-satisfaction criterion based on listening. The end user hears a difference (which may be correct) and each end user "verification" adds to the anecdotal pool but they get admonished for not producing some kind of independent validation.

 

I see possible misleading measurements and a double standard emerging. If its good for the goose it's good for the gander.

 

False equivalence, David. There's no lack of objective evidence that DISTORT does what is claimed, you are just too lazy to look for it. It generates a nonlinear transfer function to simulate a desired level of distortion. This is verifiable and measurable, and objective measurements have been shared by others. It's extremely easy to do it yourself, if you have any real interest.

 

On the other hand, there's a complete lack of objective evidence and, in fact, some pretty strong, independent objective evidence against the claims made by @superdad. Neither he, nor his engineer have produced any such evidence, and no independent testers have, either. We've gone through this before, so why do you keep rehashing it?

 

There's nothing similar between uncontrolled, purely subjective reports of "many satisfied customers" and the measurement results from independent testers with no financial interest in the product. There's no need for someone to purchase a $15k jitter analyzer to show that DISTORT works as advertised: it can be measured and validated using simple and free tools, such as REW, in about 2 minutes. 

 

 

 

[bluesman]

1 hour ago, pkane2001 said:

What I said, at least a dozen times, is that DISTORT will construct a nonlinearity and that the same non-linearity will generate HD when applied to a single tone, and IMD when applied to multiple tones. Any other conclusions again, are all in your head.

When I asked if you'd tested ("validated", a term at which you mysteriously took offense) your app to be sure it was not adding any distortions or other effects besides the desired one, you offered no answer other than that "DISTORT will construct a nonlinearity and that the same non-linearity will generate HD when applied to a single tone, and IMD when applied to multiple tones".  And you may well have said this "at least a dozen times" - I didn't count. 

 

You also ignored my request for any explanation, no matter how simple, of how you could generate harmonic or intermodulation distortion by adding a nonlinear transfer function to the signal path without producing any other effects on the signal.  Again, you ignored my question and gave the same answer.  Today, SoundAndMotion offered what I believe is a more accurate response - that "...if you want to add anomaly X, you have to be aware that the compression comes with it".  I, too, believe that you cannot add harmonic distortion to a signal without other secondary effects on it, e.g. compression, frequency response alteration, phase shift, etc depending on the exact transfer function used.  BTW, y=f(x) can be a linear equation depending on the function. That’s why I asked how you do what you do with your app.

 

I still do not understand how you can add a nonlinear transfer function to a signal chain with no resultant change in the signal except the production of harmonic / IM distortion.  You still haven't explained it, if it's even possible (which, from what I know, is not the case).  And compression is only the first unintended "side effect" that came to mind - there are many more. 

 

I'd hoped to learn something new from you - but that hasn't happened yet.  I still live in hope, but I'm not as optimistic now.  I'm sorry you choose to take umbrage at what I truly think are reasonable questions that are also very relevant to deciding if a measurement is useful or misleading.  

[Audiophile Neuroscience]

2 minutes ago, pkane2001 said:

 

False equivalence, David. There's no lack of objective evidence that DISTORT does what is claimed,

 

The "objective data" lacks independent testing beyond anecdotal reports from arguably questionable sources.

 

 

2 minutes ago, pkane2001 said:

you are just too lazy to look for it.

 

may I suggest that this and other assertions/slurs made earlier about me are in your own words, " missing some basic etiquette in communicating"

 

2 minutes ago, pkane2001 said:

 in fact, some pretty strong objective evidence against the claims made by @superdad.

 

Paul, maybe in your opinion but again using your own words "your opinion is not better than enyone else's"

 

2 minutes ago, pkane2001 said:

 so why do you keep rehashing it?

 

I don't, point to where i keep on rehashing it apart from mentioning it in a thread which cites misleading measurements , double standards and hypocrisy.

 

2 minutes ago, pkane2001 said:

There's nothing similar between uncontrolled, purely subjective reports of "many satisfied customers" and the measurement results from independent testers.

 

Let me fix that for you:

There's something similar between uncontrolled, subjective reports of "many satisfied customers" and the uncontrolled, subjective reports of measurements resulting from non-independent satisfied testers.

 

 

2 minutes ago, pkane2001 said:

There's no need for someone to purchase a $15k jitter analyzer to show that DISTORT works as advertised: it can be measured using simple and free tools, such as REW, in about 2 minutes. There's no need for someone to purchase a $15k jitter analyzer to show that DISTORT works as advertised:

 

All the easier to arrange independent testing.

 

 

 

 

 

 

[pkane2001]

 

4 minutes ago, Audiophile Neuroscience said:

 

The "objective data" lacks independent testing beyond anecdotal reports from arguably questionable sources.

 

 

 

may I suggest that this and other assertions/slurs made earlier about me are in your own words, " missing some basic etiquette in communicating"

 

 

Paul, maybe in your opinion but again using your own words "your opinion is not better than enyone else's"

 

 

I don't, point to where i keep on rehashing it apart from mentioning it in a thread which cites misleading measurements , double standards and hypocrisy.

 

 

Let me fix that for you:

There's something similar between uncontrolled, subjective reports of "many satisfied customers" and the uncontrolled, subjective reports of measurements resulting from non-independent satisfied testers.

 

All the easier to arrange independent testing.

 

 

You'll continue to argue to death that any objective result is anecdotal and from arguably questionable sources, without knowing who the sources are or what the result was. Why should anyone listen to you?

 

You're wasting my time. Talking in generalities, and not providing a single fact or objective result to back it up. Why are you posting in an objective forum, by the way?

[opus101]

15 hours ago, pkane2001 said:

This one, between AN and I, was started many years ago, long before DISTORT. By nature I'm a skeptic and so I ask questions, just like AN does. But that's where we differ: I don't ask questions just to prove the other side wrong. I ask questions because I'm really interested in finding and understanding the answer, and if I don't believe or understand the answer, I look for ways to learn and verify.  That's why I created DISTORT, that's why I created DeltaWave.

 

What I'm getting from this is that @pkane2001's motives are pure and @Audiophile Neuroscience's aren't. Since this is the Objective forum - is there like, um any evidence in support of this (implied) claim? Or did I misinterpret the implication?

[Audiophile Neuroscience]

2 minutes ago, pkane2001 said:

 

 

You'll continue to argue to death that any objective result is anecdotal and from arguably questionable sources, without knowing who the sources are or what the result was.

 

..or you could just tell us.

 

Thus far we have an anonymous guy on the internet referring to Audio "Science" review as verification and an implicit endorsement on some other guys blog with a survey of 67 self selected internet respondents that used the App (but don't even address it's accuracy that I could see)

 

2 minutes ago, pkane2001 said:

 

Why should anyone listen to you?

 

I don't, or you?

 

2 minutes ago, pkane2001 said:

You're wasting my time. Talking in generalities, and not providing a single fact or objective result to back it up. Why are you posting in an objective forum, by the way?

 

That's simple, I am asking for independent objective data from an objectivist who claims it exists. Why should that be so hard?

[pkane2001]

2 minutes ago, Audiophile Neuroscience said:

 

..or you could just tell us.

 

Thus far we have an anonymous guy on the internet referring to Audio "Science" review as verification and an implicit endorsement on some other guys blog with a survey of 67 self selected internet respondents that used the App (but don't even address it's accuracy that I could see)

 

 

I don't, or you?

 

 

That's simple, I am asking for independent objective data from an objectivist who claims it exists. Why should that be so hard?

 

The ignore list is really getting a workout today...

[pkane2001]

6 minutes ago, opus101 said:

 

What I'm getting from this is that @pkane2001's motives are pure and @Audiophile Neuroscience's aren't. Since this is the Objective forum - is there like, um any evidence in support of this (implied) claim? Or did I misinterpret the implication?

 

There's an old objective practice of tying up hands and feet and throwing the one being tested into a large, freezing body water. If they are pure of heart the water will support them and not let them drown. I suggest you start with AN. 

[Audiophile Neuroscience]

4 minutes ago, opus101 said:

 

What I'm getting from this is that @pkane2001's motives are pure and @Audiophile Neuroscience's aren't. Since this is the Objective forum - is there like, um any evidence in support of this claim?

 

I agree it was an unsubstantiated slur made by @pkane2001. I understand he doesn't like what I am saying, but I believe he feels he is right.

[opus101]

2 minutes ago, pkane2001 said:

 I suggest you start with AN. 

 

Why? I've not seen a claim, implied or otherwise about his motivations from him.