Hi-Res Music Challenge Again: Doing It Right!

[R1200CL]

I think Mark Waldrep latest project deserves its own tread. 

 

As he states:

“I'm also very open to fielding any suggestions or comments that could improve this project.”

 

http://www.realhd-audio.com/?p=6494

 

Under is a copy out the text in link above. 

It's been over a month since my last post. Time flies when there's travel (Charlie and I had a road trip to the mountains of Colorado for some late season skiing), a weekend in Big Sky with friends, and a long weekend in manning my table for the annual AXPONA show in Chicago (more about that in a future post). The big news is that I've been granted a sabbatical for the fall semester from CSU Dominguez Hills to conduct a research project into the audibility of high-resolution music.

Here's a portion of what I submitted to the University research committee:

Statement of Purpose and Description of the Proposed Project

a.  Goals of the project – The goal of the proposed research project is to test whether music consumers can reliably identify a Hi-Res Music selection versus a downconverted of the same file at CD or standard resolution using real high-resolution sources, rigorous testing methodologies, and statistical analysis. This issue is a major point of contention among audio consumers and despite previous studies remains unsettled.  

Since Edison’s invention of audio recording in the late 19^th century, it has generally been assumed that recording fidelity has steadily increased. Music consumers have been offered and purchased music on a wide variety of physical formats including crude, low fidelity cylinders, monophonic lacquer 78 rpm discs, stereo vinyl 33 1/3 rpm LPs, analog tapes, 8-track cartridges, compact cassettes, compact discs (CDs), and recently pure audio Blu-ray discs. The arrival of digital audio file formats ushered in the era of CDs, music “ripping”, and MP3 playback. Recently, Digital Music Retailers (DMR) like iTunes and Amazon Music that provided file downloads have given market share to music streaming services including Spotify, Apple Music, and Tidal. 

One of the latest initiatives being promoted by NARAS (The Recording Academy®), the CTA (Consumer Technology Association), the DEG (Digital Entertainment Group), major consumer electronics companies, and the major music labels is called Hi-Res Audio/Music or High-Resolution Audio/Music. This new digital format is the latest new format to promise additional fidelity for consumers. But does it? Does merely doubling or quadrupling the sample rate of a digital sound file and adding 8-bits to each sample deliver perceptible improvements in fidelity?

Figure 1 – The history of consumer music format – from standard-res analog to hi-res digital.

b. Preliminary Work – I have been actively involved in the production, promotion, and discussion of high-resolution audio since its introduction in 2000. My label, AIX Records, has recorded and released almost 100 high-resolution, award-winning music albums in a variety of physical and downloadable formats. I am a recognized expert in the area of high-resolution recording. I have served on boards (CEA High End Audio) and technical working groups (AES), given papers and keynote address at prestigious conferences, (CES, Las Vegas, AES Latin America, Bogota, Columbia, AES London, InterBEE Tokyo, AXPONA Chicago), written over 1000 articles/blog posts on the topic (www.realHD-Audio.Com), and consulted on a previous study for the Consumer Technology Association.

On two separate occasions, I have collaborated with online publications in casual high-res audio studies (AVS Forum and the “HD Audio Challenge”). I have critiqued and reviewed previous studies in the area and found serious flaws in their methodology and results. It’s past time to do a proper study. 

c. Specific Activities – There will be multiple phases required to complete the proposed study. They are: 1. Identify, select, evaluate, and prepare up to 20 music examples for use in the listening phase of the research, 2. Identify, qualify, and engage with up to 1000 participants across a wide demographic, 3. Distribute and collect the survey materials among participants, 4. Analyze results, and 5. Present statistical breakdown of findings. These activities can be accomplished within the 15 weeks of the leave requested. 

d. Specific Benchmarks – The proposed research will contribute to — and possibly resolve —  the ongoing debate regarding the value proposition of “hi-res audio”. The well-known Audio Engineering Society paper by E. Brad Meyer and David Moran of the Boston Audio Society entitled: Audibility of a CD-Standard A/DA/A Loop Inserted into High-Resolution Audio Playback conducted in 2007 was shown to be seriously flawed by myself and others because it didn’t use real high-resolution sources.

The proposed study will be submitted to for publication in the AES Journal, other publications (print and online), and otherwise widely disseminated among the professional audio community, audiophiles, and consumers.

Specific benchmarks will include: announcements of the study on major online audio sites, development of approved survey methodology and test materials, acceptance of test subjects, and presentation of outcomes.

e. Summary of Project – This proposal describes a study into the perceptibility of Hi-Res Audio recordings vs. downconverted versions at CD specifications. A number of music examples will be prepared in both Hi-Res and CD fidelity, distributed, and evaluated by a pool of qualified test subjects. Their feedback will be analyzed and published.

Initial Steps - Announce the Study and Seek Input

Unfortunately, there is no funding associated with the granting of a sabbatical. There will undoubtedly be costs associated with preparing, executing, and reporting this study. I am considering starting a crowdsourcing campaign to offset some of these costs. However, I don't think it's going to be expensive primarily because I am not planning on hosting playback sessions at my studio or other facility — although I am not opposed to doing so if the consensus is that will provide for a better result.

My intention is to improve and repeat the HD-Audio Challenge that I created on this site some many months ago. Over 4000 people read the blog, almost 250 downloaded the test files, and just over 100 responded with their submissions. I provided 6 different selections of music in both the original 96/24 PCM format and downconverted to 44.1/16 CD standard. Participants were encouraged to play the files on any system (speakers and/or headphones) and then let me know which one of the pair was high-resolution. The results seemed to indicate that listeners did no better than random guessing — although I did not do a rigorous statistical analysis. I will engage with statisticians for the proposed study (if you're one of those experts, please feel free to contact me).

I'm also very open to fielding any suggestions or comments that could improve this project. There will be complaints and objections to whatever I choose to do but I'm open to modifying my plans if warranted and beneficial. All of the content will come from my own AIX Records catalog because I know the catalog exceeds the fidelity of standard CDs — at least by software analysis. There will be a wide variety of music genres, ensembles, and instrumental/vocal selections represented among the roughly 20 tracks. If you know of specific tracks from my catalog that you believe show perceptible improvement over standard CDs, again please let me know.

This should be fun. I hope that this study will contribute to our knowledge of hi-res music. I believe it can.

 

[BrokeLinuxPhile]

Most people are using their phones to access the web these days, and each phone type has it's issues decoding HD audio correctly.  Everything is going to get resampled to the native sampling rate of the OS in that case.  Make it clear to those taking the test of this limitation and have them do it from a standard mac/PC.

 

Not having a control for the experiment in the form of a static system/environment that never changes will skew the results.  When HDTV first gained popularity, I would go into house after house and people were watching the game in standard def.  Always saying they never see a difference with this fancy new tech.  I changed the channel on their converter box and their jaws dropped.  They were misconfigured or misusing their setup, they just didn't know to change the channel.  This combined with a changing source file and you have two variables interacting, making data hard to comprehend.