Where Were The Science Journalists?

One thing missing from the Floyd Landis hearings at Pepperdine University from May 14th through the 23rd was a science journalist to translate much of the detailed scientific arguments into plain English for the masses. Setting aside the sensationalized aspect of the trial, which has no real bearing on whether or not synthetic testosterone was found in Floyd Landis’ urine sample from Stage 17 of the 2006 Tour de France, the hearings boil down to whether or not some complicated scientific tests were performed properly, and whether the data was correctly interpreted.

So it’s no great surprise that one big flaw in USADA’s case slipped under the radar last week. And that flaw was summed up at Trust But Verify this way:

Most people, including me, did not get the importance of the methodological peak identification problems that Brenna nearly perjured himself to try to bury. LNDD has a designed-in mistake in their SOP for IRMS that is common among all tests they have ever run. They don’t run a calibration mix that includes the 5a, 5b, and pdiols, so they can’t identify peaks in the IRMS correctly, or in conformance with the ISL IDCR — creating an ISL violation on any AAF they have ever declared on 5a or 5b – pdiol. This is a disaster for LNDD and whoever has said their procedure was correct — including Brenna, Ayotte, Catlin, Aquilera, and Botre.

OK, for the technically less inclined, what does this mean? In calibrating their equipment, LNDD doesn’t run a standard (or known) sample that contains all of the metabolites they’re looking for in a testosterone screening. If memory serves me correctly, the sample LNDD used only contained one of the metabolites.

Now, the tricky thing about this testing is that you need to match a number of peaks on a graph for a known compound to peaks on a graph for the unknown compound (that would be Landis’ sample, in this case) in order to identify what the sample contains. If they line up, then you’ve got a match. If they don’t, then no match.

But if you’re matching against only one peak, and trying to interpret whether or not the other peaks are in the right places, then you are going to have an error-prone system. See, the thing is that the IRMS instrument at LNDD tends to go out of specification easily, according to Dr. Simon Davis, an expert on these particular instruments. (How expert? Davis used to install and configure the very same instrument as the one used at LNDD for Landis’ tests.) Since the machine can go out of spec so easily, LNDD needs to be careful to ensure that the instrument is running properly and adjusted properly.

That being the case, you can’t rely on matching a single peak to make a firm identification. Think of this example:

You have two thermometers that you’ve determined both read -40 degrees* when the temperature outside is really, really cold. Think Siberia in the winter time. Or certain parts of the American West or Midwest. That’s tongue freezing to a flagpole cold, if you were so foolish as to do something like that.

Now, since the thermometers agree on this temperature, would you be correct in assuming they agree all the time? No, you wouldn’t. If you knew they were measuring on the same scale, then they might stay calibrated through their entire range. But if you’re not measuring temperature using the same scale, then your results will differ.

So how do you know whether the thermometers are actually calibrated for the same scale. See if, at other temperatures, they agree. So, for example, at -40, water will be frozen (for the most part, I’m going to skip over vapor pressure to make the discussion simpler). At a certain temperature ice melts into a liquid, and at a higher temperature water turns into a gas.

So let’s look at the problem of whether you can correctly identify something based on a measurement system that agrees at a single point. Using one of the two thermometers, you determine that ice, which is quite solid at -40 degrees, becomes a liquid at 0 degrees and then becomes a gas at 100 degrees. Now, using the other thermometer, you take an unknown solid and determine that it is a solid at -40, a liquid at 0 and a gas at 100.

Have you correctly identified the unknown as water? No. Why not? Because you didn’t calibrate the second thermometer. You only know that it agrees at one particular point along its scale. But you don’t know if it’s measuring on the same scale or another scale. Before interpreting the results for the unknown solid, you need to use the second thermometer to measure those properties for a known sample of ice/water.

If the results match up, then the interpretation that the unknown solid is ice would be more, well, solid. But if one thermometer measures temperature on the Celsius and the other measures temperature using the Fahrenheit scale, then these would clearly be two different substances. Why? Because 0 degrees F and 100 degrees F are cooler temperatures than O degrees C and 100 degrees C.

Now, this example is a bit different than what’s going on in the IRMS system, it’s just meant to illustrate why it’s important to calibrate your measuring instruments correctly. You could also think of tuning musical instruments. If one string of a guitar is in tune with the same string on a different guitar, are the instruments in tune?

Not necessarily. All of the other strings could be out of tune, which would make for quite a racket if two musicians were trying to play the same song on those guitars in unison.

Going back to LNDD and the Landis case, here’s the problem in a nutshell: By not running a known sample that contained all of the metabolites of interest, how can one be confident that what LNDD claims to have found was, in fact, what they said it was? You can’t. Even if the peaks for the other metabolites are approximately where they should be, if you have nothing to match them up against, you don’t know that the substance they’re analyzing is what they say it is.

So if LNDD can’t prove that they’re really analyzing the metabolites of testosterone, all of their results are meaningless. They (and we) don’t know what they were looking at. So any conclusions drawn from the data would be pure speculation, and not anything of value. If this is the way LNDD has been doing their work, then as TBV noted, all of their work is suspect.

And that, of course, means the case against Floyd Landis is suspect. This revelation will likely be the bellwether of how thoroughly the arbitration panel considers the evidence presented over the last couple of weeks. If they ignore this and find against Landis, then you can be pretty sure that they’ve decided the case based on other things — the famed “non-analytical” positive that Dick Pound so dearly loves to use to hang suspected witches — er, I mean dopers.

* True fact: The Celsius and Fahrenheit scales coincide at -40 degrees.