The ABX test scores audibility, not preference

The tool moment

Before a musician can ask whether an AI mastering pass, a lossy export, or a plugin's vintage mode is actually audible, someone had to build a way to test that claim without letting expectation do the deciding. Engineer David Clark described such a method in a 1982 paper in the Journal of the Audio Engineering Society, on high-resolution subjective testing using a double-blind comparator, building on a 1981 JAES paper by Stanley Lipshitz and John Vanderkooy on subjective evaluation. The Boston Audio Society's own account describes the resulting hardware, the ABX comparator, as a box with three buttons, A, B, and X, that lets a listener switch freely among a known source A, a known source B, and an unlabeled X that the device has randomly assigned to match one or the other.

What the documents show

The Boston Audio Society's description of the ABX comparator states that on power-up the device generates a hundred random assignments of X to A or B, and that a listener's task is to identify which one X matches, with the correct answer withheld until the trial is scored, making the test double-blind. The society's page gives a concrete statistical threshold: across 24 trials, 17 correct identifications is the bar for 95 percent confidence that a listener detected a real difference rather than guessing. A companion page describing related testing conducted at the University of Waterloo in February 1984 credits the same 1981 and 1982 JAES papers as the method's origin. These are the society's own historical pages rather than the original JAES papers, which were not accessed directly for this note.

What stays with the musician

An ABX result answers one narrow question, whether a listener can reliably tell X apart from A and B at better than chance, not whether the audible difference matters musically, and not which of the two options sounds better. A musician still decides whether a detected difference is a flaw worth fixing or a character worth keeping.

Judge it by listening

Editorially, before accepting a claim that a plugin's bypassed and processed states, or two export formats, are indistinguishable, it is worth asking whether that claim survived an ABX-style blind trial or only a sighted comparison where the listener knew which was which.

• Was the comparison blind, or did the listener know which source they were hearing?
• Did enough trials run to clear a real statistical threshold, rather than a single guess?
• Does a detected difference change the musical decision, or only confirm that one exists?

The ABX comparator did not settle audio's oldest arguments. It gave them a scoreboard, which is a more useful thing to bring into a mixing session than another opinion about what someone thinks they hear.