Jump to content
The Computer Audiophile

Misleading Measurements

Recommended Posts

@bluesman I like the way you approach finding answers. You pick up your guitar, record it, and make your points. Using different tools, I often take a similar approach.

 

I have said for years, I'm an open-minded skeptic (similar to the Sagan quote above). So, a bit of the friendly skeptic part: I don't (yet!) see evidence of sum and difference frequencies that you mention, other than the linear effect of beats. Your recording doesn't help me (yet). Can you help me? I'd like to work through this with you to understand the sum and difference vs. IMD issues.

 

Some comments on your recording:
- I don't like the Spectrum function in Audacity (which I otherwise love), so I use my own.
- I don't find any "notes" (frequencies) near 246Hz
- I find 2 peaks at about ~221+ and ~226+Hz, giving the noticeable ~5Hz beats. I looked at the 2nd, 3rd and 4th harmonics: they are there, but I don't see anything at (F1 + F2) frequencies.
- There is not any signal near 5Hz (F2-F1)
- There's a lot of "stuff" between 20-34Hz, with a peak at 26+Hz. I have no idea about the origin.
- "Something" changes at about 1.0s. It is audible and visible as a sudden amplitude drop. Any ideas? Could you have adjusted your hand, or...?
- All the "stuff" around 26Hz is gone after 1.5s
 

Share this post


Link to post
Share on other sites
3 minutes ago, pkane2001 said:

But I already mentioned the ASR thread on testing and validating of DISTORT while it was being developed (still is, really) easy to look up and find what testing was done and what results were produced by me and by others. 

Hi Paul,

I understand, have seen the other thread, and will have some questions for you there.

But here, keep in mind you and @Audiophile Neuroscience are stuck in the same place I am with my cousin. I'm waiting for her to reach out to me, and she is doing the same in reverse. You want AN to put in some effort (define, read/search, try out), while he is waiting for you to put in effort to provide evidence of validation. It won't work. Either one of you has to flinch first (I won't hold my breath) or others will need to jump in. I attempted above.

Cheers, SAM

Share this post


Link to post
Share on other sites
2 hours ago, pkane2001 said:

Welcome to the loop (get out while you still can :))!

Hi Paul,

 

I understand your perspective, and given the complexities of online relationships, I don't wish to convince you of anything

 I've interacted a few times with AN also, even before "the big break", sometimes very satisfyingly. He does have a few issues upon which he chomps down like a pit bull and won't let go - even if you turn the hose on him. Like validation. If it is done in good faith, he reminds me of a couple of reviewers I've dealt with a few times. They have issues they chomp down on, but even after a bit of swearing and name-calling during the rewrite/resubmit phase, I have to admit that the final product (the paper) was better after dealing with the "issues".

In this thread, the question of good faith arrises, and I hope giving the benefit of the doubt is warranted.


SAM

Share this post


Link to post
Share on other sites
7 hours ago, Audiophile Neuroscience said:

 

You seem to be saying that this App equates to using a hammer or it's truth is self-evident, no objective data required, correct?

 

.....I see our posts have crossed and now you are saying something different about hammers! Objective data is useful.You have changed tacks to pre-requiste knowledge.....

 

Yet So, Paul who invented the App says his Apps "do exactly what they were designed to do and have been validated through independent testing".

 

So, in an objective forum I am asking any objectivist (or non objectivist) for the objective data that tells me it does indeed  "do exactly what they were designed to do and have been validated through independent testing".

 

This is separate and distinct from understanding how the App does it, or any other specific knowledge, or how to use it. "Independent testing" is by its very nature , well, independent of my understanding or knowledge and does not rely on my expertise or lack thereof, to validate the App.

 

Is that so hard to understand?🤷‍♂️

 

I suggest we have an intermission to let others have a say (if they're interested)

 

 

 

You'll have to learn how to code an application to understand how it works, this is out of scope of the thread and "irrevelant" for now. 

 

Like it has been said earlier the app has already been "validated" and verified . Unless you have the expertise too objective data about how it bas been code will be "useless" for this goal of this thread.

 

And sure I'll change my position whenever a rationnals arguments will be provided. It includes things that I can think of in the future.

 

So for OP, objective data are always welcome.

 

 

Share this post


Link to post
Share on other sites
4 minutes ago, SoundAndMotion said:

Hi Paul,

 

I understand your perspective, and given the complexities of online relationships, I don't wish to convince you of anything

 I've interacted a few times with AN also, even before "the big break", sometimes very satisfyingly. He does have a few issues upon which he chomps down like a pit bull and won't let go - even if you turn the hose on him. Like validation. If it is done in good faith, he reminds me of a couple of reviewers I've dealt with a few times. They have issues they chomp down on, but even after a bit of swearing and name-calling during the rewrite/resubmit phase, I have to admit that the final product (the paper) was better after dealing with the "issues".

In this thread, the question of good faith arrises, and I hope giving the benefit of the doubt is warranted.


SAM

 

Agreed, SAM. I've had more problems getting anything specific or helpful out of my conversations with AN, but I'm willing to consider that I am 50% responsible for such an outcome :)

 

Share this post


Link to post
Share on other sites
1 hour ago, Patate91 said:

 

You'll have to learn how to code an application to understand how it works, this is out of scope of the thread and "irrevelant" for now. 

 

Like it has been said earlier the app has already been "validated" and verified . Unless you have the expertise too objective data about how it bas been code will be "useless" for this goal of this thread.

 

And sure I'll change my position whenever a rationnals arguments will be provided. It includes things that I can think of in the future.

 

So for OP, objective data are always welcome.

 

 

 

Welcome to audiophile style! I think it’s a great capability to be able to consider other perspectives and to change position if sensible evidences are provided. Opened minded is the word for it.

 

To be blunt, I don’t think that your examples with planes and hammers are related to the issue/phenomenon that is discussed here, and more important doesn’t make your perspective stronger or easier to grasp. I would even say that it has the opposite effect.  

Share this post


Link to post
Share on other sites
1 minute ago, Summit said:

 

Welcome to audiophile style! I think it’s a great capability to be able to consider other perspectives and to change position if sensible evidences are provided. Opened minded is the word for it.

 

To be blunt, I don’t think that your examples with planes and hammers are related to the issue/phenomenon that is discussed here, and more important doesn’t make your perspective stronger or easier to grasp. I would even say that it has the opposite effect.  

 

Distort app is a tool, hammer is a tool. All objective data for a tool cannot be published.

 

Same thing goes for plane, we don't have access to all the objective data before we buy a ticket. There's no doubt or skeptism to have about the plane's abilities to fly and bring you from point A to B.

 

But people can still ask for it if needed. Manufacturers or creators will provide it or not. For our situation feel free to ask everything toi want, we'll  see what Pkane anwser.

 

Again objective data can only be misleading if you don't have the knowledge to interpret it.

Share this post


Link to post
Share on other sites
Just now, The Computer Audiophile said:

That's the rub. Nobody has the knowledge of everything.

 

Sure, but that's not a "problem".

 

Irrational buying habits is a problem.

Share this post


Link to post
Share on other sites
16 minutes ago, Patate91 said:

 

Distort app is a tool, hammer is a tool. All objective data for a tool cannot be published.

 

Same thing goes for plane, we don't have access to all the objective data before we buy a ticket. There's no doubt or skeptism to have about the plane's abilities to fly and bring you from point A to B.

 

But people can still ask for it if needed. Manufacturers or creators will provide it or not. For our situation feel free to ask everything toi want, we'll  see what Pkane anwser.

 

Again objective data can only be misleading if you don't have the knowledge to interpret it.

 

Which type of objective data that’s relevant varies greatly between a plane, a hammer and an app for measuring. Which data that are of importance depends, and I think it’s safe to say that the data/measurements that we needed or want to know for a simple device like a hammer and a complex thing like an airplane is not comparable.  

Share this post


Link to post
Share on other sites
1 minute ago, Summit said:

 

Which type of objective data that’s relevant varies greatly between a plane, a hammer and an app for measuring. Which data that are of importance depends, and I think it’s safe to say that the data/measurements that we needed or want to know for a simple device like a hammer and a complex thing like an airplane is not

 

In fact I don't even need the hammer's data to use it and find that it does what's supposed to do.

 

Now what objective data do you expect from a tool like Distort? Seems that no one want to anwser it.

Share this post


Link to post
Share on other sites
9 minutes ago, The Computer Audiophile said:

I believe it's a problem for consumers trying to weed through buying options.

 

With respect to buying habits, I will never judge another person for what s/he purchases. 

 

Being irrationnal is not a judgment, at different degree we are all irrationnal and full of biais.

 

If a buyer has to weed out option there won't be free ticket, he'll have to inform himself, simple has that. If a buyer doesn't want or doesn't have time to learn and fer informed he'll have to trust sellers and or manufacturer. At this point objevtive data, even a "pass" /no pass is irrevelant

Share this post


Link to post
Share on other sites
32 minutes ago, The Computer Audiophile said:

That's the rub. Nobody has the knowledge of everything.

 

Knowledge is a familiarity, awareness, or understanding of someone or something, such as facts (propositional knowledge), skills (procedural knowledge), or objects (acquaintance knowledge). Knowledge can be acquired in many different ways and from many difference sources, including but not limited to experience, reason, memory, testimony, scientific inquiry, education, and practice. The philosophical study of knowledge is called epistemology.

 

https://en.wikipedia.org/wiki/Knowledge

 

Share this post


Link to post
Share on other sites
50 minutes ago, Patate91 said:

In fact I don't even need the hammer's data to use it and find that it does what's supposed to do.

 

Now what objective data do you expect from a tool like Distort? Seems that no one want to anwser it.

I fear you're all missing the issue in questioning the concept of validation, especially given how it's being used in this thread.  I was the one who first raised the question, and what I asked repeatedly (still without an answer) is whether and how we know that the app in question does ONLY what it's said to do.  Some participants are turning this into a bizarre debate over a non-issue. I did not and am not questioning whether it does what its creator says it does - as has been pointed out, that's easy to validate.  What we still don't know is what else it may be doing to the signal passing through it.  That is not an unreasonable thing to want to know.

 

It's simply hard to believe that any transfer function applied to a source file would do absolutely nothing but add one specified anomaly to that file.  The analogy to medication is a sound example of the fact that few if any interventions into anything have only one effect.  In most cases, additional effects are unintended and often undetected prior to widespread use.  Many common medications like Vioxx were "validated" through FDA mandated clinical trials, only to be found after marketing to cause serious cardiac complications that resulted in withdrawal from the market.  The Boeing 737 Max was "validated" for use by the FAA, and we all know how that worked out.

 

Even a hammer can have undisclosed and/or unknown issues that potentially affect its function and its users.  I once bought an inexpensive 16 ounce framing hammer at a local hardware store while visiting friends far from home, to help them build a bench.  Halfway through the project, I struck a 10 penny nail (unhardened) and the hammer's handle broke just below the point at which the shaft entered the metal head.  When I looked closely at it, I saw that the wood was "secured" in the head by driving an old fashioned cut nail into the end of it to widen it and wedge it in.  That cut nail was too large, driven in well off center and penetrated below the hammer's head, weakening the shaft and causing the break.  I always wear safety glasses when working, so the pieces of splintered wood that flew off didn't cause me any harm - but they could have.  I might also have questioned what kind of wood the handle was made of, since it was obviously too soft and easily splintered for the use to which it was put.

 

So, specifically, what I'd like to see is a list of the other effects on the source signal that were considered possible and shown not to occur with use of the app in question.  This is a very reasonable thing to ask - it's neither pejorative nor argumentative.  It's just a factual inquiry that deserves an answer.  Since I didn't get one and I want to try the app myself, I'll try to determine the answer myself.  And I promise to try hard to avoid misleading measurements.

Share this post


Link to post
Share on other sites
5 minutes ago, bluesman said:

 What we still don't know is what else it may be doing to the signal passing through it.  That is not an unreasonable thing to want to know.

 

 

[snip]

 

 

So, specifically, what I'd like to see is a list of the other effects on the source signal that were considered possible and shown not to occur with use of the app in question.  This is a very reasonable thing to ask - it's neither pejorative nor argumentative.  It's just a factual inquiry that deserves an answer.  Since I didn't get one and I want to try the app myself, I'll try to determine the answer myself.  And I promise to try hard to avoid misleading measurements.

I guess you didn’t see post#776. If you answer mine, I’ll answer yours.

Share this post


Link to post
Share on other sites

This link is for a research paper that contains the parameters used to estimate IMD analytically for a Class D amp case, and is compared to a Matlab DSP simulation, and also measurements on the actual amplifier:

 

https://core.ac.uk/download/pdf/162670379.pdf

 

Can this case be used as a test case for DISTORT? This would help to stop the infinite-loop discussions about what DISTORT actually is doing...

 

Also, as far back as 2010, there were publications about a Distortion Analysis Toolkit which is Matlab based. Is DISTORT doing anything different than this available software, or Matlab DSP functions for instance?

Share this post


Link to post
Share on other sites
7 minutes ago, SoundAndMotion said:

I guess you didn’t see post#776. If you answer mine, I’ll answer yours.

I read it - I just don’t see anything in it that relates to my question in any way at all.  Confirming the presence of desired effects is part of validation.  So is confirming the absence of undesired effects.  Surely a skilled, experienced, knowledgeable, creative engineer could think of at least one effect of adding a nonlinear twist to that straight wire with gain.

 

There’s a lot of excellent audiology literature on the effects of nonlinear processing on sound quality, most of it from research into hearing aid design and function.  Compression is very well studied as a nonlinear distortion of auditory input - and it’s well known and validated with good solid metrics that compression affects perceived sound quality.  So one might reasonably ask if an app that adds nonlinear processing to a vocal or musical data stream changes its dynamic range at all.  If it does, how?  Is it an across the board attenuation? Is it frequency related? Amplitude related? Random?

 

If I haven’t made my point by now, I don’t think I ever will. Enjoy your debate.

Share this post


Link to post
Share on other sites
1 hour ago, bluesman said:

I fear you're all missing the issue in questioning the concept of validation, especially given how it's being used in this thread.  I was the one who first raised the question, and what I asked repeatedly (still without an answer) is whether and how we know that the app in question does ONLY what it's said to do.  Some participants are turning this into a bizarre debate over a non-issue. I did not and am not questioning whether it does what its creator says it does - as has been pointed out, that's easy to validate.  What we still don't know is what else it may be doing to the signal passing through it.  That is not an unreasonable thing to want to know.

 

It's simply hard to believe that any transfer function applied to a source file would do absolutely nothing but add one specified anomaly to that file.  The analogy to medication is a sound example of the fact that few if any interventions into anything have only one effect.  In most cases, additional effects are unintended and often undetected prior to widespread use.  Many common medications like Vioxx were "validated" through FDA mandated clinical trials, only to be found after marketing to cause serious cardiac complications that resulted in withdrawal from the market.  The Boeing 737 Max was "validated" for use by the FAA, and we all know how that worked out.

 

Even a hammer can have undisclosed and/or unknown issues that potentially affect its function and its users.  I once bought an inexpensive 16 ounce framing hammer at a local hardware store while visiting friends far from home, to help them build a bench.  Halfway through the project, I struck a 10 penny nail (unhardened) and the hammer's handle broke just below the point at which the shaft entered the metal head.  When I looked closely at it, I saw that the wood was "secured" in the head by driving an old fashioned cut nail into the end of it to widen it and wedge it in.  That cut nail was too large, driven in well off center and penetrated below the hammer's head, weakening the shaft and causing the break.  I always wear safety glasses when working, so the pieces of splintered wood that flew off didn't cause me any harm - but they could have.  I might also have questioned what kind of wood the handle was made of, since it was obviously too soft and easily splintered for the use to which it was put.

 

So, specifically, what I'd like to see is a list of the other effects on the source signal that were considered possible and shown not to occur with use of the app in question.  This is a very reasonable thing to ask - it's neither pejorative nor argumentative.  It's just a factual inquiry that deserves an answer.  Since I didn't get one and I want to try the app myself, I'll try to determine the answer myself.  And I promise to try hard to avoid misleading measurements.

 

To validate the application you have to use it, and as said earlier Pkane already asked other people to validate it.

 

If I create an algorithm that do addition. Then if you enter 2 and 2 and it gives you 4 I guess you'll know it works correctly. No need for external validation.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now



×
×
  • Create New...