Maybe spending stupid amounts of money on power cords isn't entirely harmless

[Jud]

But if the confidence interval around any measurement is +/-2 db with phone-based systems, remembering that this includes repeatability as well as accuracy, a true difference of less than 2 will not be identified consistently and accurately - those 10 points may be off by as much as 4 db from each other from measurement to measurement on the same phone or among 10 different phones. And a difference of 3 or more doesn't require measuring to detect - all but the seriously hearing impaired would hear it.

 

It's the directionality as much as or perhaps more than the difference. If the differences were an artifact of the error bands, you'd expect directionality and difference to be reasonably random within those bands. If directionality and difference are consistent across multiple measurements, random error begins to look less and less like a good explanation. It's analogous to the principle behind CD rips not being certain to be perfect, but if you get two that are identical between two different CD players it's pretty much certain they are both perfect (though of course the latter is a comparison across a few zillion bits rather than, say, 10 points on a recording).

[wgscott]

But if the confidence interval around any measurement is +/-2 db with phone-based systems, remembering that this includes repeatability as well as accuracy, a true difference of less than 2 will not be identified consistently and accurately - those 10 points may be off by as much as 4 db from each other from measurement to measurement on the same phone or among 10 different phones. And a difference of 3 or more doesn't require measuring to detect - all but the seriously hearing impaired would hear it.

 

Two separate considerations are accuracy and precision. The true value might be 6 db, but if the detector consistently measures between 3.9 and 4.1, for example, the precision is much better than 2b. So differences will have an error of approx. ±0.4 db, even if the absolute value of the measurement is off by more.

[bluesman]

Two separate considerations are accuracy and precision. The true value might be 6 db, but if the detector consistently measures between 3.9 and 4.1, for example, the precision is much better than 2b. So differences will have an error of approx. ±0.4 db, even if the absolute value of the measurement is off by more.

 

That's all true. But the NIOSH test results show enough variance in both to make reliable detection of a 1 db change in SPL impossible with a phone app. All of the apps and phones tested were orders of magnitude away from a confidence interval of +/- 0.1 db and none even achieved +/- 1 db for accuracy or for consistency. As a 3 db level change would be clearly audible to anyone with unaided serviceable hearing, and even a 2 db change should be audible to most (even if falsely perceived as a quality change), controversy over whether or not a change in level actually occurred means that if it did, it wasn't more than a db or so. So it's not very likely that any app on any mobile phone would help identify the very slight changes about which this thread concerns itself.

[wgscott]

about which this thread concerns itself.

 

Even though it took to page 46 to get back to this.

[Jud]

That's all true. But the NIOSH test results show enough variance in both to make reliable detection of a 1 db change in SPL impossible with a phone app. All of the apps and phones tested were orders of magnitude away from a confidence interval of +/- 0.1 db and none even achieved +/- 1 db for accuracy or for consistency. As a 3 db level change would be clearly audible to anyone with unaided serviceable hearing, and even a 2 db change should be audible to most (even if falsely perceived as a quality change), controversy over whether or not a change in level actually occurred means that if it did, it wasn't more than a db or so. So it's not very likely that any app on any mobile phone would help identify the very slight changes about which this thread concerns itself.

 

In other words, one would be very unlikely to get, for example, a comparison of 10 points on two iPhone recordings where Recording A consistently measured ~.4dB higher than Recording B. But in the unlikely event one *did* obtain such a comparison, it would be likely the data resulted from actual volume differences rather than random variation.

 

So may as well try and see if you get lucky.

 

Edit: BTW, I'm getting a 404 Not Found at your NIOSH link.

[bluesman]

So may as well try and see if you get lucky.

 

Edit: BTW, I'm getting a 404 Not Found at your NIOSH link.

But how would you know whether you were lucky and correct or (thousands of times more likely) unlucky and misled by your data?

 

For some reason, that link doesn't work today (I did check it in the post before submitting). Here's the URL, which does work right now: CDC - NIOSH Science Blog – So How Accurate Are These Smartphone Sound Measurement Apps?

The fully qualified URL is http:// blogs.cdc.gov/niosh-science-blog/2014/04/09/sound-apps/ (remove the 2 spaces after the forward slashes - every time I try to paste it, it turns into a link)

[Jud]

In amongst the data from NiOSH (from the 2014 test) are some not too shabby results:

 

The results reported in Table 2 show that the SPLnFFT app had the best agreement, in unweighted SPLs, with a mean difference of 0.07 dB from the actual reference values.

(My emphasis.)

[bluesman]

In amongst the data from NiOSH (from the 2014 test) are some not too shabby results

But look at the width of the confidence intervals. Had they reported both mean and median, the variance in outliers would have been evident in a large difference between those two parameters (because the median is less sensitive than the mean to outlier values in the range). Unfortunately, they didn't test multiple phones, each with the same app, to look at repeatability in the real world. Unless the p value is < 0.05 on an adequately powered analysis of the A and B measurements, measured differences are less than 95% likely to have occurred because of the intervention of interest. In this case, it means that "measured" changes in SPL could not reasonably be attributed to changes in cables rather than to randomness in the hardware, software etc being used to measure.

[esldude]

Spl apps aren't going to do it. But recordings might. Record the demo tracks while having your phone stationary and in portions without crowd noise you'll have a chance of determining a genuine loudness change. Or even EQ changes over limited frequency ranges. Especially if several people in different parts of the crowd do this.

[Jud]

Spl apps aren't going to do it. But recordings might. Record the demo tracks while having your phone stationary and in portions without crowd noise you'll have a chance of determining a genuine loudness change. Or even EQ changes over limited frequency ranges. Especially if several people in different parts of the crowd do this.

 

If variability is a function of the app rather than the mic, yes. But in the NiOSH study, the mic was more variable than the apps.

[bluesman]

Spl apps aren't going to do it. But recordings might. Record the demo tracks while having your phone stationary and in portions without crowd noise you'll have a chance of determining a genuine loudness change. Or even EQ changes over limited frequency ranges. Especially if several people in different parts of the crowd do this.

 

If variability is a function of the app rather than the mic, yes. But in the NiOSH study, the mic was more variable than the apps.

Face it, guys - there's simply no way to accurately and reliably measure a change of 1 db or less in mean or peak SPL (weighted, unweighted, iced, glazed or salted) at an open demo session. It'd be hard enough to do it with a Bruel & Kjaer 2270 in a controlled environment, and it simply won't happen with a phone app.

 

A change of 3 db would be heard by everybody in the room. So if some hear a level change and others don't, and it's really there, it has to be in the range of 1 db or less.

[Priaptor]

Face it, guys - there's simply no way to accurately and reliably measure a change of 1 db or less in mean or peak SPL (weighted, unweighted, iced, glazed or salted) at an open demo session. It'd be hard enough to do it with a Bruel & Kjaer 2270 in a controlled environment, and it simply won't happen with a phone app.

 

A change of 3 db would be heard by everybody in the room. So if some hear a level change and others don't, and it's really there, it has to be in the range of 1 db or less.

 

And face it given our ages we are lucky to just be able to hear

[Jud]

face it, guys - there's simply no way to accurately and reliably measure a change of 1 db or less in mean or peak spl (weighted, unweighted, iced, glazed or salted) at an open demo session. It'd be hard enough to do it with a bruel & kjaer 2270 in a controlled environment, and it simply won't happen with a phone app.

 

A change of 3 db would be heard by everybody in the room. So if some hear a level change and others don't, and it's really there, it has to be in the range of 1 db or less.

 

ok.

[esldude]

If variability is a function of the app rather than the mic, yes. But in the NiOSH study, the mic was more variable than the apps.

 

It appears these box plots are being misread. In one set they are showing the spread of the apps with a given phone. In another the spread of the phones. Neither indicates that a given phone will give one reading now and one 2 db different 10 minutes from now. Also they did not calibrate any apps and many have that capability.

 

 

I can test a couple phones out later, I don't know why a microphone would change its response over time except for time and weighting in the apps or for ALC in recording apps. That is why some recording apps that record to wav and don't auto adjust level will be needed, and could work well enough for this purpose. There are such apps available. I used one at one time and two phones seprated by a few feet to do a stereo recording just for kicks. I had to roll off treble and bass as the phones were two different models, but between 100-8000hz it sounded pretty okay.

[bluesman]

It appears these box plots are being misread. In one set they are showing the spread of the apps with a given phone. In another the spread of the phones. Neither indicates that a given phone will give one reading now and one 2 db different 10 minutes from now.

That's not the two I posted below - one is for unweighted measurements and one is for A-weighted measurements for different apps on the same phones. But, as I said earlier, they didn't measure inter-test variance for the same devices, which is what you're addressing too.

 

There are many reasons for inter-test variance in the same device. Factors like ambient temperature, pressure, vibration, etc affect the output of a measurement mic. As Jud points out, directionality affects measurement accuracy - a minor change in positioning of a handheld phone/SPL meter can change the reading by a few db, as can a minor change in the position of one person standing or sitting close to the measuring device. Under controlled circumstances (e.g. tripod mounting in an audiology booth), I doubt that a given iPhone would vary by even one db from measurement to measurement. But with a handheld phone in an audio demo room amidst a crowd, it'd be almost impossible to maintain consistency.

[wgscott]

Even if you could bring six absolutely error-free SPL meters into the demonstration, this won't prove much if anything.

 

Playing amateur detective never works well.

 

A much simpler and open approach might be to bring your own CD, and have a skeptical user control the audio system.

 

Then you test whether audible differences in power cords persist.

[Jud]

That's not the two I posted below - one is for unweighted measurements and one is for A-weighted measurements for different apps on the same phones. But, as I said earlier, they didn't measure inter-test variance for the same devices, which is what you're addressing too.

 

There are many reasons for inter-test variance in the same device. Factors like ambient temperature, pressure, vibration, etc affect the output of a measurement mic. As Jud points out, directionality affects measurement accuracy - a minor change in positioning of a handheld phone/SPL meter can change the reading by a few db, as can a minor change in the position of one person standing or sitting close to the measuring device. Under controlled circumstances (e.g. tripod mounting in an audiology booth), I doubt that a given iPhone would vary by even one db from measurement to measurement. But with a handheld phone in an audio demo room amidst a crowd, it'd be almost impossible to maintain consistency.

 

Just wanted to note that wasn't the sort of "directionality" I meant. I was talking about directionality of the difference between two sets of measurements of demos. In other words, when comparing measurements of demo A at 10 different times (e.g., at the start of the demo and once every 5 seconds for the first 45 seconds) with measurements of demo B at the same times, if the actual dB levels of the demos were quite close and the app and/or phone were giving you unreliable readings, you'd expect demo A to measure louder at times and demo B to measure louder at times. If demo A measured louder all 10 times, then one could reasonably conclude demo A was louder.

[bluesman]

Just wanted to note that wasn't the sort of "directionality" I meant. I was talking about directionality of the difference between two sets of measurements of demos. In other words, when comparing measurements of demo A at 10 different times (e.g., at the start of the demo and once every 5 seconds for the first 45 seconds) with measurements of demo B at the same times, if the actual dB levels of the demos were quite close and the app and/or phone were giving you unreliable readings, you'd expect demo A to measure louder at times and demo B to measure louder at times. If demo A measured louder all 10 times, then one could reasonably conclude demo A was louder.

True enough. Depending on the magnitude of the differences, you could run a 2x10 chi square on the 10 member data sets using the "louder-softer" dichotomy, to see if there was a significant difference between them.

[Jud]

True enough. Depending on the magnitude of the differences, you could run a 2x10 chi square on the 10 member data sets using the "louder-softer" dichotomy, to see if there was a significant difference between them.

 

I can tell you what I've done with the SPL graphing app I mentioned: I see whether there's a direction in the graphed SPL difference between two files at multiple points. If there is, I adjust the volume of one file until the graphed differences, if any, are small and random in direction.

[tranz]

Or you could try an amazingly sensitive set of measurement tools...

[Jud]

Or you could try an amazingly sensitive set of measurement tools...

 

Were you referring to...?

 

[Sal1950]

Were you referring to...?

 

[ATTACH=CONFIG]26390[/ATTACH]

Yep, when involved in a bias controlled overseen blind listening tests.

[Solstice380]

Despite all the negative stuff I still like Santa Fe.

 

I lived in Albuquerque for a little more than 10 years. We called SF "the city different". That was even before the mass Californication.

 

The biggest downside of New Mex for me was that eventually you complain about >340 days of sunshine per year. You feel like you should always be outside doing something with very few rainy days to listen to music and read! However, the east side of the Sandias is pretty lush and green. Otherwise, learn to appreciate the various shades of brown out there. I still tell everyone that THE best 7-10 day road trip in the U.S. is the loop around the 4 corners.

 

Jud, sounds like you may be up near Bernalillo or Placitas area, or a little west of there. We built our house in Corrales by the Rio Mud. We had irrigation rights so we had a lawn. Otherwise we would have done the Xeriscape route. There are some new areas since I left in 1999. I still go to ABQ and Los Alamos for biz. Enjoy! (And hunt down Sadie's Cocinita on 4th Street in ABQ for great New Mex food!)

[tranz]

Yep, when involved in a bias controlled overseen blind listening tests.

 

And how would you like to do that during a public demo at an audio show?

[Allan F]

You call it avoidable because you create a BS scenario that doesn't exist and claim it does exist to claim it is an error so you can sue and perpetuate the crooked system you are engulfed in strictly for self gain.

Another example of you displaying your ignorance by resorting to uninformed opinion and insults instead of facts. Just a few samples:

http://www.who.int/gpsc/tools/faqs/evidence_hand_hygiene/en/

Infection prevention in hospitals: The importance of hand hygiene | Infectious Disease News

Hand hygiene: Back to the basics of infection control

My final comment on the subject. Over and out.