Fit For Purpose? Or Not?

I was gearing up to write a post on the politics site, when I saw this comment by strbuk on a social networking site that I frequent:

ARRRRGGGGHHHHHH!!!!!!!!!! http://tinyurl.com/amobuh.

OK, I thought, what’s up with that? So I clicked the link and it took me to a story on the USA Today web site with the headline Swiss study calls testosterone tests into question. Stephen Wilson, an Associated Press sports writer reports:

A key test for catching drug cheats in sports should be scrapped because it fails to take into account vital ethnic variations, according to a study by a leading Olympic anti-doping lab.

The study, published in the British Journal of Sports Medicine, claims the testosterone test is “not fit for purpose” and should be replaced by an athlete “passport” system that tracks a competitor’s biological patterns.

Interesting. Very interesting. The study, conducted at the WADA-accredited lab in Lausanne, Switzerland looked at the urine steroid profiles of 171 professional football (soccer in these parts) players and came to the conclusion that a one-size-fits all approach using the testosterone/epitestosterone (T/E) ratio doesn’t work, due to wide variations in the naturally occuring T/E level across ethnic groups. The study found that Hispanic athletes had the highest threshold to suggest abuse (5.8) and that Asian athletes had the lowest (3.8). African and Caucasian athletes had ratios similar to the Hispanic athletes, at 5.6 and 5.7, respectively.

“These results demonstrate that a unique and non-specific threshold to evidence testosterone misuse is not fit for purpose,” the study said.

WADA’s current threshold, a T/E ratio of 4, clearly doesn’t work, given this news. But the article says WADA is “well aware” of ethnic variations. So yoou might ask, “If WADA knew of ethnic variations, why did they insist on creating a single standard? Or, for that matter, why did they use the T/E ratio at all?” Good question. The agency goes on to say that the T/E ratio is “just one of several warning signals” which are followed up by carbon isotope ratio (CIR/IRMS) testing that is “not affected by genetic factors.” The CIR/IRMS may be affected by other factors, such as diet, which WADA is also well aware of, but which they didn’t mention. (Perhaps Wilson didn’t know to ask.)

One of Lausanne’s own researchers tells the AP’s reporter that given the study’s results, there is no one T/E threshold value that can be applied with any accuracy.

“Even if we apply different thresholds we will not be sensitive enough in detection of testosterone and we have to use another system,” Christophe Saudan, a researcher with the Lausanne lab, said in a telephone interview. “Now we have to turn to the biological passport which takes into account the individual range of the athlete.

“The athlete is his own reference. We have to look at the athlete’s reference rather than a population reference.”

The passport system would measure an athlete’s baseline blood and urine profiles against numbers gleaned from later tests. Such “longitudinal” tests wouldn’t necessarily look for specific substances, but could detect changes in body chemistry that would indicate use.

FIFA’s chief medical officer Jim Dvorak says that of 25,000 drug screenings performed each year over a five year period, only 51 turned up positive results for anabolic steroids. For the mathematically inclined, assuming that Wilson reported the test numbers correctly, that’s a positive rate of — are you ready? — 0.0408 %. Now, if he meant that the 25,000 tests were the total number carried out over the 5 year period, the percentage goes up to 0.204 %. Either way, precious few positive results. What does Dvorak have to say about all of that?

“In our experience over the past five years we have not found any single one T/E ratio between 4 and 6 which has proven to be a positive case,” Dvorak said. “We see that there are ethnical differences.”

Dvorak, who is a member of WADA’s medical committee, plans to present the study’s results to the WADA committee at their next meeting in Montreal in September. Arne Ljungqvist, who heads the IOC’s medical commission, was rather blase in his response to the study’s results, telling Wilson that the study merely confirmed what anti-doping experts already knew. He also said it didn’t cast doubt on any previous cases.

“Using T/E ratios is fairly accurate and useful as a screening method but it is never the final proof,” Ljungqvist said.

True. By WADA’s own protocols, the T/E ratio is not enough to convict an athlete of steroid abuse. But what Ljungqvist didn’t address (and may not have been asked about) is whether the results from T/E screening tests may induce cases of experimenter bias when the more advanced testing is performed. That is, do the results of the CIR/IRMS testing frequently tend to confirm the original suspicion, or do they accurately reflect whether an athlete actually used synthetic testosterone or other anabolic steroids.

Questions about the standards (or lack of common standards) for what constitutes proof of such use when CIR/IRMS testing is done makes me wonder whether or not that test, too, is fit for purpose. It may be. But having seen how the testing was performed in one rather famous case, I don’t know that all labs are equally qualified and capable of making such determinations.

Finally, if you read through the article, you probably already saw the reference to Floyd Landis. (You didn’t seriously think I would pass up commenting on that, did you?) I’m not sure how much Wilson, the AP reporter, followed Landis’ case, but the reference to Landis’ T/E ratio perpetuates an old canard. Only one of several test results from Landis’ A sample showed such a ratio. The rest were in the 5 – 6 range. In fact, it was the inconsistency in his initial T/E results that caused the Pepperdine arbitration panel to throw those results out when considering his case. Repeating that number merely reinforces wrong information and wrong impressions that have been passed along over the last several years. Numbers do lie from time to time.