So About Those Test Results…
It’s been a good several weeks since Selena Roberts’ and David Epstein’s article The Case Against Lance Armstrong was published by Sports Illustrated, in both online and print forms. For those who aren’t committed fans of cycling, their article lays out in great detail a saga of doping, or alleged doping, by the 800-pound gorilla of American cycling. Well, not that Lance actually weighs 800 pounds. But he does wield quite a bit of power both in North American cycling circles and at the international level. And I’m not talking about the kind of power that one could measure with a PowerTap or similar type of training device, either.
Lance Armstrong certainly doesn’t need any help defending himself against these most recent allegations. And I’m not about to take a position on his guilt or innocence. But it’s worth looking at the Sports Illustrated article, because at least part of the article illustrates a problem in how certain types of articles are reported.
Roberts and Epstein may be close to whatever the truth of Armstrong’s athletic feats happens to be, at least when it comes to the big picture. So many people have told similar stories about the world-famous cancer survivor that there must be some grain of truth to what these people are saying. That, or there’s a massive anti-Lance conspiracy designed to take the 7-time Tour champion down a few notches.
I have a hard time believing that so many people would conspire in such a manner, but stranger things have happened. Especially in the world of professional cycling. Even though the article may come close to the heart Armstrong’s seemingly legendary athletic feats, there is at least one part of the article where the reporters wade into the weeds. That’s the part where they present what seems like compelling scientific evidence that yes, Virginia, our esteemed hero of the Grande Boucle committed the heinous act of doping.
What I’m referring to is this:
From 1990 to 2000, Armstrong was tested more than two dozen times by Catlin’s UCLA lab, according to Catlin’s estimate. In May 1999, USA Cycling sent a formal request to Catlin for past test results—specifically, testosterone-epitestosterone ratios—for a cyclist identified only by his drug-testing code numbers. A source with knowledge of the request says that the cyclist was Lance Armstrong. In a letter dated June 4, 1999, Catlin responded that the lab couldn’t recover a total of five of the cyclist’s test results from 1990, 1992 and 1993, adding, “The likelihood that we will be able to recover these old files is low.” The letter went on to detail the cyclist’s testosterone-epitestosterone results from 1991 to 1998, with one missing season: 1997, the only year during that span in which Armstrong didn’t compete. Three results stand out: a 9.0-to-1 ratio from a sample collected on June 23, 1993; a 7.6-to-1 from July 7, 1994; and a 6.5-to-1 from June 4, 1996. [emphasis added] Most people have a ratio of 1-to-1. [ed. Not exactly true. Most white males of western European background have this ratio. Men — and women — from other parts of the world can exhibit higher natural ratios than what is suggested here by the writers.] Prior to 2005, any ratio above 6.0-to-1 was considered abnormally high and evidence of doping; in 2005 that ratio was lowered to 4.0-to-1.
What the casual fan needs to know about the testosterone test is this: When an athlete’s A sample is tested, the lab actually subjects at least three portions of that sample to the T/E ratio test. A sample only comes up positive if three results occur in the same general ballpark, give or take a test’s measurement uncertainty (which we’ll come back to shortly).
[Catlin] said that because he tested by code and not by name, he has “no clue which sample belonged to Lance,” but he admits the data are disturbing. He explains that one failed confirmation would be a “once-in-a-blue-moon” occurrence.
Interesting assertion, and Catlin certainly is one who knows from drug testing. But I don’t know that the assertion holds up. Revisiting a the Landis case, his original A sample T/E ratios were all over the place. Sure, you probably heard about the 11-to-1 ratio that came up in the A sample, but there was also a reading above 14-to-1 and several in the range of about 5-to-1 or 6-to-1. (The results on the B sample were more consistent, however, but that’s a different story.)
The point is, fluctuations occur. How often they occur is something we don’t get a glimpse of, because when results are reported, we almost always see a single ratio, rather than the set of ratios reported.
As for the three high T/E ratio results detailed in the letter, [Catlin] says, “that’s very strange.” When [the] letter was read to [Andreas] Breidbach recently, he too expressed concern, saying, “Wow, that should not happen. If you find a nine and can’t confirm, then something is very wrong with your screening test.”
There’s an interesting point. How good is the screening test that’s performed? Unfortunately, the article doesn’t really tell us. It could well be that Breidbach is inadvertently letting us all in on a big secret: the screening test is not very good. But an actual admission that the testing isn’t good would make the testers look, well, not very competent. And that would cast the whole anti-doping system in a rather bad light.
The Roberts/Epstein article gives us single results for the tests in question, and those results paint a certain picture. But the ratios, themselves, are meaningless without context. The ratios give an idea of the magnitude of the difference between the concentration of testosterone and epitestosterone in a sample. What they don’t tell us is the raw numbers. Were the testosterone numbers elevated? Or were the epitestosterone numbers depressed? That has a bearing on whether you can even begin to suggest that an athlete has been doping, or whether something else is going on in the person’s body.
The numbers present, though, lead to the conclusion that on those occasions, Armstrong was up to his eyeballs in testosterone. Maybe he was, maybe he wasn’t. Since other tests of the same sample didn’t provide enough certainty about the results, the labs involved didn’t declare any adverse findings.
Of the three results the article discusses, one very likely falls into the gray area of measurement uncertainty. Measurement uncertainty is, when you take a measurement of something, how close is the measured value to the real value. It’s expressed in terms of plus or minus X, and depending on how the measurement uncertainty was determined, there is a certain probability that the true value lies within the range of values specified. Another term for this is confidence interval, from the world of statistics.
In the case of the lab testing Floyd Landis’ infamous sample in 2006, the measurement uncertainty was ± 0.8. While the Sports Illustrated article doesn’t tell us the measurement uncertainty for any of the three results attributed to Lance Armstrong’s samples, we can make a somewhat educated guess that the 6.5-to-1 result was probably within the bounds for the top end of the allowable T/E ratio back when it was conducted.
From that point of view, the explanation of why a 6.5-to-1 ratio wouldn’t count as a positive is pretty straightforward. The other two, we don’t have enough information about to draw a conclusion.
By presenting the T/E ratios, the writers give the impression that Armstrong tested positive for testosterone on three occasions. But, according to the rules in place and the way the testing is performed, he didn’t test positive. The reality is that these particular results don’t prove much of anything, and are a weak argument at best that Armstrong was not riding the straight and narrow path to cycling glory.
The more compelling part of the story is that a number of people seem to paint a consistent picture — and not a pretty picture — of Big Tex. While that all falls into the realm of he said/she said, it’s the consistent appearance of smoke from a number of quarters that leads one to suspect that there may be a fire going on somewhere. How big a fire is yet to be seen.
And speaking of bombshell articles…
A week ago, Paul Kimmage published an interview of Floyd Landis in The Sunday Times (London, that is). And then he put a transcript of the interview out at NYVelocity.com. (Hat tip to AustinCyclist for the links.) The transcript clocks in at about 31,000 words and can take a good several hours to read. It gives further context to Landis’ comments in The Sunday Times, and adds additional information, too. Kimmage does a very good job interviewing his subject (he spent seven hours with Landis in November conducting the interview), but like all interviews, some responses beg for follow-up questions.
Sometimes, which questions those happen to be depend on what we bring to reading an interview, and what knowledge we have of the subject already. Even reporters can go back to their own work and find that they missed a follow-up question or two. Still, despite its imperfections, the lengthy transcript of the Kimmage/Landis conversation is one of the best interviews I’ve read in a long while, and certainly one of the best interviews of a professional cyclist that I’ve seen.
What raised my eyebrows were the sections that discuss the — for lack of a better phrase — endemic corruption in the halls of the UCI’s headquarters. It seems as though the people heading the organization view their power as a license to shake down anyone and everyone they can. Just as some commenters here have suggested over the years. And, the leadership of the UCI basically appears to have the attitude, “it’s our way or the highway.” AustinCyclist also provides this link to Cyclocosm.com, where documents backing up Landis’ story about his difficulties collecting from bank guarantees administered by the UCI can be found. The guarantees are required of all UCI-licensed teams, to ensure that riders on teams that collapse will still get paid.
Meanwhile, more stories are surfacing suggesting that former UCI chief Hein Verbruggen was/is a corrupt old sod, and that he ran the UCI more as his personal fiefdom than as a professional organization. Predictably, the UCI’s current head has said, basically, “No corruption here. Move along.” Or, to get the quote right:
“It’s impossible to be corrupt in the way we’re being accused, in terms of bribery and assisting riders cover up doping positives,” McQuaid told Cyclingnews.
“We’d welcome any investigation into the UCI. There has never been corruption in the UCI.”
Be careful what you wish for, Mr. McQuaid. If an investigation did happen, you might be proved a liar. Investigators and reporters have a way of digging up awfully embarrassing information. Just ask former Senator [[Gary Hart]].
About testosterone tests of Lance, is missing the fact that Lance should have tested positive (false positive) while is cancer was on.
Lance raced and won Tour Dupont and has been tested, if we want to believe that USOC was not corrupted too, or didn’t drop samples.
What did happen there?
Kimmage did a great interview with Landis, a part of the system is uncovered but we have more questions.
We can only hope that more honest journalists will ask the good questions to UCI and will investigate the dark side of cycling management (teams, organisers, federatons and UCI).
Considering both the rather wide range of T/E values reported for the Landis A samples in 2006, and now the putative Armstrong samples, I have a feeling the test is not all it is cracked up to be. I recall some statements made back in 2006 or 2007 in discussing of the Landis case that the test was being used in a manner it was not designed to.
There was also some questioning as to how a confidence interval should be interpreted in respect to a cut-off point. Should a test be deemed positive if the cut-off point falls within the CI, or should the whole interval be required to be past the cut-off point, or perhaps it is sufficient if the centerpoint is beyond the cut-off point? My somewhat elementary understanding of statistics doesn’t provide an answer and I tend to think that any answer would depend on the particular situation. For example, in a test screening for possible cancer, you probably would go with an indication, ever so slight, as a “positive” requiring additional testing. But in the context of anti-doping, this is harder to answer. Is it better to let a possible doper off in order to reduce the likelihood of convicting an innocent athlete or is it better to possibly punish an innocent athlete in order to be sure we get as many dopers as possible?
Given the severity of penalties currently applied, I go with the idea of giving the benefit of the doubt to the athlete.
Jean,
Been a while. I was wondering about whether Lance should have tested as a false positive while the cancer was growing. Interesting point. As for the USOC being corrupt, well, there’s certainly some history there, if you get my drift.
William,
Think about the confidence interval this way: How certain can you be that the measured value is the true value? For a measured value X, with a confidence interval of X plus or minus Y, if the level of confidence is 95%, then 19 times out of 20, the true value will lie within X – Y and X + Y. Unless you have a known value to start, you can only say that the true value is somewhere within that range.
Now, suppose that the test which came up at 6.5-to-1 has a measurement uncertainty of plus or minus 0.8. The interval within which the true value will occur (for whatever degree of confidence) ranges from 5.7 on the low end to 7.3 on the high end. Or, put another way, part of that range of values is “non-positive.”
To be completely certain (at least to whatever degree of probability is implied in the confidence interval) that the true value exceeds the 6.0-to-1 threshold value, the measured value would need to exceed 6.8. Otherwise, the range of possible true values includes some for which a positive result can’t be declared, as per our “hypothetical” example.
Or, to put it another way, if the range of possible true values includes some “non-positive” results, you need to give the athlete a certain amount of benefit of the doubt.
For anyone who might consider Kimmage to be a responsible or objective journalist, you can give this interview a listen: http://bit.ly/eGKfuC.
Some quotes from the interview: KImmage called it “ludicrous” for the UCI to have referred Contador’s case to his national cycling federation (ludicrous for UCI to follow its own rules?). He called Contador’s press conference following the announcement of a one-year ban a “ludicrous spectacle” — particularly odd, given that this interview took place the day before the press conference. Contador’s contaminated meat excuse is “a joke” and Contador is a “cheat” who has committed a “heinous sin” against cycling. When Kimmage was reminded that Contador had threatened to quit if he was sanctioned, Kimmage said repeatedly “Good riddance. Good riddance to him.” He also said that Andy Schleck’s comments about Contador still being the winner of the race indicated that Schleck had probably doped too.
It’s an entertaining interview, until you realize that Kimmage is supposed to be a responsible journalist, with a certain amount of objectivity, and that we are relying on this guy to report the true story of Floyd Landis. Kimmage is not a journalist; he is an angry guy with a specific agenda he’s trying to pursue against his enemies in cycling. He may be fighting a good fight, but he’s fighting a fight, not reporting the news. If you want to get an idea why this guy asked Landis the questions he asked, and failed to ask so many other questions, listen to the interview.
Larry,
Good points. You’ll notice that I didn’t actually call Kimmage a journalist. He’s definitely someone with strong opinions about doping and cycling — and those strong opinions could easily get in the way of objectively reporting the facts of a story.
He’s more in the tradition of those reporters who come to an issue with their point of view already formed, and who search out stories that illustrate that point of view. Nothing wrong with that, as long as the writer/reporter/columnist/”journalist” doesn’t misrepresent how he approaches his work.
I wouldn’t be surprised if he avoided asking questions that might provide answers which wouldn’t confirm his own point of view. He’s bringing his own experiences in the peloton, and his view of what goes on there, to his writing, after all. The transcript seems pretty complete for the amount of time he and Landis spent in conversation. Thirty one thousand words spread over 7 hours is just over 4600 words per hours. Just shy of 80 words per minute.
Still, even taking into account Kimmage’s point of view, the transcript published is the lengthiest interview of Landis that I’ve seen, and it certainly adds to the picture of what kind of person Landis is and what he’s experienced in the world of professional cycling. Not a complete picture, because that would be nigh on impossible. But closer, perhaps, than we’ve seen up to now.
Rant:
I do know what a confidence interval is, the question here is how to interpret a test result. We might well require that the entire X plus or minus Y interval fall within the range required for a positive finding in order to convict an athlete. That would mean that athletes whose range overlapped in part the positive range (and hence also in part the negative range) would go free when there was some possibility that they indeed were guilty. Or we could require that the entire confidence interval fall in the negative range in order to reach a negative finding. In the first case, we run the risk of a false positive; while in the later we run the risk of a false negative. Or we could split the difference is some way, perhaps using the X value alone to make the determination while perhaps using the size of the confidence interval to make some sort of judgement whether or not the test results were “good enough” to use. In other words, the Y value has to be within certain limits; otherwise we throw out the test results.Which way to go, in my opinion at least, is more of a philosophical question than a mathematical one. Do we want to risk possibly letting dopers off in order to be reasonable sure we are not falsely punishing innocent athletes or do we accept some collateral damage in order to maximize the number of dopers we nail. I personally would lean more to the idea of letting off some dopers in order to avoid falsely punishing innocent athletes, but I am sure there are those who feel that the problem is so serious that we should error in the other direction.
William,
Sorry about that. Certainly, if we require that the entire range be above the threshold, then we have a pretty high degree of confidence that the result is a true positive test. And, as far as judging the reliability of a test, if the Y value is too large, it doesn’t make the test look very good, so throwing out the results would be appropriate.
I think you’ve nailed it when you said that the question is more a philosophical one. It’s similar to the question often asked in the context of criminal prosecutions: Is it better to let a guilty person go free to avoid putting an innocent person in jail, or is it better to put an innocent person in jail rather than risk letting a guilty one go free?
That describes the difference between the American approach to justice and the approach in some other countries. It seems as though it also describes the differences in approach to anti-doping cases, too.
I think a lot of the debate as to whether it’s best to err on the side of the innocent or the guilty is all a matter of WHO is making that decision.
It’s quite easy for lawmakers (and in this I include the WADA lawmakers) to sit in their easy-chairs and decree that setting standards somewhere in the middle (where there will eventually be collateral damate to an innocent) is worth the cost in the long run to NAIL all the guilty parties. HOWEVER, when YOU (or someone in your family or a close friend) IS that collateral damage person…all of a sudden things would be quite different.
In the criminal prosecution world, innocents DO go to jail. Their life and liberties have been wrongly taken from them, and nothing can be done to get that back, even if they end up being exonerated. In sports, a false positive can totally ruin a career. At the very least casue you to lose a few years in the prime of your very short sporting life sitting on the sidelines all because YOU were the one who got caught in the fringes of the net searching for the REAL guilty ones.
I like to think that in our modern society that we would WANT to err on letting guilty go free to INSURE no innocents are convicted…but in reality we all know this isn’t how things work. Be it criminal prosecution or doping prosecution, there are agenda’s and politics running amok, and most assuredly getting in the way of the TRUTH. Just look at our criminal justice system to see that finding THE TRUTH isn’t necessarily the objective.
I rather enjoyed the full transcipt of Floyds interview. Sure, there are many thing I would like to ask that come to mind after reading it. But it shed him in a new light to me, and I quite honestly have abandoned my anger towards him and now just hope he can figure out what to do with the rest of his life and maybe find some peace. I think it let me see the ‘human’ side…and how easy it would be to get on the ‘wrong road’ and be unable to reverse course. Surely he has faced some extremely difficult decisions in his life, as we all do. And part of being human is making mistakes and hopefully learning from them.
Matt,
No doubt, no system is perfect and innocent people will be punished or sent to jail. And that has a serious impact, as you point out, on those people’s lives. My own preference is to err on the side of caution. We can always hope that a guilty person who manages to escape punishment will learn something from the episode and change his/her ways. That may not happen, but we can hope it does. What we can’t do is give back a person the lost part of his or her life, the lost opportunites or easily restore a damaged reputation when someone is wrongly punished or sent to prison.
As for the transcript of Kimmage’s interview with Floyd, I did enjoy reading it (although it did take a while). I agree, it gives a window into the human side of Floyd, and the consequences of the choices he made. I sure hope that he won’t be confronted by the same kinds of issues and choices in the future as he was in the past.
Here’s a link about Ricardo Ricco becoming ill from doing his own blood transfusion:
http://edition.cnn.com/2011/SPORT/02/09/cycling.ricco.contador.doping/#
Will this frighten others into NOT doing blood doping? I won’t bet on it….
I also finished Kimmage’s interview with Floyd, well worth the time to read every word. Shows how lies can snowball out of control.
Speaking of “bombshells”, this could prove interesting or not – as usual no real details from the media yet but I imagine JeffN just might have spoken with these guys when he was in Europe recently. that or he’s booking another flight:
For years, the underground firm International Pharmaceuticals supplied athletes around the world with the latest performance-enhancing drugs, helping it attain cult status in the doping scene and make millions. Now investigators have raided the firm’s headquarters in a German village. Its customers apparently included a number of top athletes whose identities could now be revealed.
http://www.spiegel.de/international/germany/0,1518,744401,00.html
MikeG
I noticed the article mentions that lists were recovered with first names, but no mention of full names being found. While a name like “Lance” or “Alberto” might be interesting and arouse suspicions such would be far from conclusive. Of course, the IP people might decide to fill out information (assuming that they themselves have such knowledge), or it may be that the authorities are being coy about the nature of information they have.
Mike and William,
I, too, wonder what some of those first names were. Oscar? Alberto? Alejandro? Riccardo? Lance? I guess as the case develops, we’ll learn more. Definitely looks to be Europe’s Balco equivalent.
Barbara,
I guess Ricco’s situation proves that one doesn’t need to be smart to be a professional athlete. Whether he took someone else’s blood or he used his own, improperly stored blood, his hospital stay was entirely avoidable. But as someone once said, “Stupid is as stupid does.”
Another good read on Ricco: http://www.cyclingnews.com/features/opinion-ricco-the-perfect-pariah
The bottom line, these folks injecting blood, risk wise its no different than taking Heroin or similar and OD’ing or contracting Aids from it, etc.. risk vs. reward.. the reward associated must be pretty insane, not something normal folks can comprehend.
Well, recall the informal poll of Olympic candidates a few years back, where many of them said they’d take something for a guaranteed gold medal today, even if it meant they’d die in 10 years. And then there’s all the “recreational” drug users, who do pot or coke or smack or whatever and the closest they ever come to athletic competition is watching the Super Bowl.
I sometimes think that all the advances in medicine, with cures and/or immunizations for many diseases which once claimed a heavy toll have led us to believe that there’s a pill for everything.
White Rabbit.
One thing that really bothers me about Floyd’s assertions (well, there’s more than one, but this one sticks out in my mind since I had blood drawn last week) and comes to light again w/ Ricco’s plight: if so many of them are doing blood transfusions as frequently as Floyd makes it out to be, WHERE are all the needle sticks?
Obviously there must be places that aren’t out in the open (ie: the typical vein on the arm) or they’d be easily seen and be a dead giveaway. When I donate blood or have blood samples drawn, I don’t care how good the nurse was… I get a nice bruise and scab for at least a day or 2 (and I have MONSTER veins that a blind person could hit).
It’s not like druggies injecting themselves w/ a tiny little hypodermic needles under the nails and such (brrrr…that gives me shivers). Even a small IV needle leaves a scab/mark of some sort. And the risk getting a noticable big scab/bruise would be HUGE I’d think if it were out in the open.
Just wondering. Anybody have any insight here? I just don’t see how it’s possible to be SO prevalent in the peleton when their chosen sport attire is typically bare arms/legs.
Well, my experience is a bit different. I give blood and while it would be fairly easy to detect the puncture up close, I do not get any bruising or scab. I doubt that there’s anything noticeable from the sidelines or the like. Of course, if I was doing this more often than every 56 days, it might be different. Whether or not UCI/WADA officials are visually inspecting riders, I don’t know. And things could perhaps be “explained” as IV rehydration or such.
Then there’s always veins available in other, hidden locations. I was in a hospital once and had an IV hooked up in a vein in the back or my hand. It would have been nicely hidden under a typical cycling glove. Ditto veins on the foot or in the groin or the like.
AC,
Ricco does make the perfect pariah these days, doesn’t he? Clearly, he won’t win any contests for the highest IQ in the peloton. If you’re gonna dope, at least learn how to do it without killing yourself in the process.
Matt/William,
I vaguely remember a story about an oft-accused but never proved doper who, it is said, is quite good with using makeup to conceal his tracks — in more ways than one.
William, I also have had an IV hooked up to the vein on the back of my hand once and thought of that (the cycling gloves would cover it nicely)…however, what about your own teammates? (and your director, coach, etc)? hanging out at the hotel, after the race, before the race. You’d look awfully silly walking around w/ gloves on all the time. And unless the ENTIRE team actually IS doing it (and everybody would pretend not to see whatever evidence might be visible), you’d be at risk for that one time you did bruise or scab and let your guard down for the wrong moment. It just seems that even w/ makeup or hidden locations, EVENTUALLY the wrong person would see some kind of a mark (massage therapist, chiropractor, etc)…they may be employed by the team but sooner or later something will be said to somebody else and things will come out. The only way to keep a secret is for NOBODY else to know. That would sure be a lot of paranoia and fear to live in. But I guess that’s what they (the dopers) would be living anyway. Sheesh…just another reason I never went pro (yuk yuk…like THAT was ever gonna happen!)