Which Standard Would You Choose?

I’ve probably opened up a can of worms by speaking out about who I think should oversee the certification of the anti-doping labs, but I’m going to stick with it, at least for one more night. The International Standards Organization publishes a number of standards, among them ISO 17025 which according to their web site

[S]pecifies the general requirements for the competence to carry out tests and/or calibrations, including sampling. It covers testing and calibration performed using standard methods, non-standard methods, and laboratory-developed methods.

It is applicable to all organizations performing tests and/or calibrations. These include, for example, first-, second- and third-party laboratories, and laboratories where testing and/or calibration forms part of inspection and product certification.

and ISO 15189, which according to the ISO website

[S]pecifies requirements for quality and competence particular to medical laboratories.

Some of my comments over at TBV inspired this reaction from Ferren, who said

That’s ridiculous. The anti-doping tests are employee drug screens, not medical tests. They undergo the same level of scientific vetting as all employment based drug tests, which is publication of the research in peer reviewed journals. Same as the DOT drug screening methods, which are much more important than sports PED testing.

As for the accreditation, WADA adds their specific accreditation to the ISO/IEC 17025 guidelines. This tightens the quality assurance. A big part of that is the performance evaluation samples that WADA expects the labs to analyze. I’d doubt there are any commercial enterprises that offer sports doping PED samples. It’s a good thing for WADA to ensure that the labs meet specific sports PED certification. It’s a relatively tiny field.

As for WADA being audited itself. Check out the Independent Observer reports it has made public.

Before I go much further, I’m going to make this offer to Ferren, whoever you may be: If you wish to write a rebuttal to what I’m going to say, feel free. I’ll be happy to post it here for you.

So here’s how I see it. Ferren is right that WADA builds on ISO 17025, adding their additional requirements to certify labs as being competent for anti-doping work. But I’m not comfortable with that as basis of the standard.

As someone who’s been subject to random drug testing at the whim of my employers over the last 10 years, I’d feel much more comfortable about those tests if they were performed by a lab that’s been certified to meet the standards for medical labs. Why? Because, by the description, ISO 17025 sounds like it could apply to labs that test electronic components, auto parts, tires or any number of other products, too. I want to know that the lab performing tests is up to snuff as a medical lab. I’m not terribly concerned whether they’re capable of testing auto parts.

Ferren may see these tests as merely employee drug screening, but I disagree. By their very nature, they are medical tests, performed to determine the presence or absence of something in the human body. Yes, they may be similar to drug screening tests, but they are more than that, too. A number of these tests involve some very demanding testing techniques where there are many opportunites for mistakes or misinterpretation.

Regarding the scientific vetting of employee drug screening tests, I don’t know where Ferren lives, so perhaps it is different in his/her country. But in the US, just being published in a peer-reviewed journal doesn’t guarantee that the testing method or the theory behind it is correct. Even if the basic theory is approximately correct, without extensive research one may not know the threshold at which to establish a violation.

The EtG test (which I wrote about here), often used to determine whether people being monitored have used alcohol, turns out to have had the threshold set too low. Some of the companies using this test say that any presence of EtG is indicative of drinking, when it’s possible to show small levels due to exposure to Purell, mouthwash, over-ripe fruit, or a number of other sources.

Dr. Gregory Skipper, who developed the test, studied a small group of samples during his initial research and wound up setting a threshold level that now turns out to be too low. His work may have been peer reviewed, but no one looked into other potential sources of EtG to determine whether his initial threshold was correct. Later, Dr. Skipper discovered that hand sanitizers such as Purell could produce readings several times the threshold value, while alcoholic beverages would produce readings up to 60 times his initial threshold value.

Peer reviewed? Yes. Needed more research? Yes. Have people been punished based on the original cutoff values? Absolutely. Were some of these people innocent? Very likely.

That’s just not good enough. If my job is on the line (and it is whenever I have to submit to one of these blasted tests), I want to know that these tests don’t produce false positives and that they do indicate what they claim to. Low level EtG readings, as Dr. Skipper notes, may be a warning sign, but they should not be considered proof.

Bottom line (for me, anyway) is that I think these tests need go through more than just publication in a peer-reviewed journal. They must also go through the full regulatory approval process.

As Ferren suggests, it’s good that the labs be certified on specific procedures needed for anti-doping testing, and WADA’s requirement that labs analyze unknown samples to prove competence is a good idea.

Do any commercial labs provide such samples and test kits? Honestly, I don’t know. But I’m sure that some companies could find a way to do so, and make money in the bargain. With several hundred thousand (and rising) anti-doping screening tests being performed each year, this could be a lucrative market for an enterprising organization. It’s a small field now, but it seems to be growing by leaps and bounds.

WADA may come closer to what’s needed by adding on to ISO 17025, but I think the system has a long way to go. I’m still firmly of the opinion that an outside, independent organization should oversee the certification of anti-doping labs. It adds greater credibility to the system to do so. The International Standards Organization has technical committees that draft the standards, and I believe that they should consider doing so for anti-doping labs, and perhaps include drug screening labs as part of the standard, too. (And if WADA wants to participate on the technical committee, that’s certainly fine by me.)

When it’s my job on the line, I want to know that the labs doing the testing meet the toughest, most rigorous standards. If I were given the choice of which lab performs the random screenings, you can bet I would choose the one certified to the medical labs standard. I’m sure — given the choice — that you would, too.