A few nights ago, the CBS News program 60 Minutes ran a story that should serve as a cautionary tale to those who would allow the science of the anti-doping system to go untested and unchallenged during hearings to determine whether an athlete committed a doping violation.
The story has nothing to do with cycling, in a direct way. But it does illustrate how an untested theory can become an accepted (legally and forensically) method, and how that method can be used to wrongly convict people of crimes they may not have committed.
The test in question, performed only at the FBI’s crime lab in Quantico, Virginia, is something called “bullet lead analysis” and it operates under the theory that the lead in bullets has a unique signature. That signature, in turn, can be used to match bullet fragments found at the scene of a crime to other bullets. And, in making the match, the test was said to be so accurate that it could identify bullets from the same box of ammunition from a single batch of bullets manufactured in a single factory.
There was one problem: The test’s theory had never really been tested. Still, it was used over a period of forty years to convict a number of people of various crimes around the country. As Steve Kroft’s story reports:
The FBI first used bullet lead analysis while investigating the assassination of John F. Kennedy, trying to match pieces of bullets discovered at Dealey Plaza with bullets found in Lee Harvey Oswald’s rifle.
By the 1980’s, the FBI was routinely using this analysis to link bullet fragments found at a crime scene with bullets found in the possession of a suspect, almost always in cases where more reliable ballistics tests were impossible.
“And could you run like a standard ballistics test on this?” Kroft asks William Tobin, a former chief metallurgist for the FBI.
“No,” Tobin says. “They’re too deformed for the conventional ballistics examinations.”
Tobin says the Quantico lab was the only place in the country that did bullet lead analysis, and the assertion that you could actually match a bullet fragment to a specific batch or box of bullets went unchallenged for 40 years — until Tobin retired in 1998 and decided to do his own study, discovering that the basic premise had never actually been scientifically tested.
“FBI lab personnel testified that you could match these fragments to this bullet,” Kroft remarks.
“Yes, that’s correct,” Tobin says.
Asked what he found out, Tobin tells Kroft, “It hadn’t been based on science at all, but rather had been based on subjective belief for over four decades.”
“So what you’re saying is that this is junk science?” Kroft asks.
“That’s correct,” Tobin says. “It’s worthless as a forensic tool.”
Strong words from someone who once was on the inside of the lab that was performing those very tests. The story is a real eye-opener. Had it not been for a retired metallurgist, who had the gumption to stand up and say “the Emperor isn’t wearing any clothes,” the bullet lead analysis test might still be in use today. And more people might be behind bars, convicted at least in part by faulty evidence.
As the story concludes:
It’s impossible to know right now how many other bullet lead cases like this there may be out there in the netherworld of the criminal justice system. But Dwight Adams, the former FBI lab director who [like William Tobin] began to have doubts about the science, believes it’s time for the government to reveal all of the information.
“I don’t believe there is anything that we should be hiding,” Adams says. “I believe that everything should be made available in regards to the testimonies and the cases that were worked involving bullet lead analysis.”
The FBI ultimately agreed, acknowledging that some of its bullet lead testimony was misleading and that it should have done more to alert defendants and the courts. As a result of the 60 Minutes-Washington Post investigation, the bureau has already reviewed nearly a hundred cases, and according to the Innocence Project there are problems with the FBI testimony in almost half of them.
There’s no way to know whether any (or how many) of the testing methods currently used by the anti-doping agencies have the same problem as the FBI had with bullet lead analysis — that of being a seemingly logical, but untested, theory. And, as closed a system as the anti-doping system is, it will take someone like William Tobin to come along and publicly out any faulty tests currently in use.
We depend on labs such as the FBI’s crime lab for rock-solid science, so that when someone is convicted of a crime, we can be reasonably certain that he or she did what the government alleges. When the evidence that comes from such a lab turns out to be lacking, it casts doubt on how well other tests from the same place might actually work.
In a similar vein, we depend on the anti-doping labs to do rock-solid scientific work, so that once someone is branded a doper, we can be reasonably sure that is the case. But as has been apparent in several high-profile cases over the last few years, not all labs function at the same level of quality. Which makes it hard to have complete faith in the results.
One can only hope that if there are any testing methods in use by the anti-doping agencies that are as scientifically murky as bullet lead analysis, it won’t take forty years before someone from inside the system has the courage to stand up and tell the rest of us that there’s a problem.
Nice topic Rant! Hard to believe for DECADES the FBI has been using bogus science to put people in jail (we are sure this is the guy, so we will make our test results match that…hmm…sounds a BIT like what we have going on in WADA if you ask me!) So now do they go BACK and let a bunch of people out of prison if it appears they were convicted on this junk science? (doesn’t mean they WEREN’T guilty, just that one piece of evidence doesn’t hold water).
I am still amazed that WADA hasn’t consolidated the worldwide lab procedures into ONE book of directives/procedures/criteria. How on earth in this day and age can one WADA lab have different testing procedures and criteria for a positive than another? It just blows my mind, yet it is STILL that way! (as has been pointed out over the last 2 years, the test results at LNDD that branded Floyd as positive wouldn’t have been so at the Australia and UCLA labs). I would think WADA would have been terribly embarrased after all the info that was exposed into the light of day by Floyd and his team, and they should be stepping up, making all their procedures and criteria public, so they can be tested and accepted as proof-positive, and able to withstand scientific scrutiny! Until this is done, I have a hard time believeing ANYTHING that comes from WADA!
Matt,
Thanks. It is hard to believe that the FBI was using that technique for 4 decades before someone spoke up about it. The fact that WADA hasn’t truly standardized their practices and required all labs to meet those specific standards just flabbergasts me, too.
Hi Rant, Just reading this article http://sports.yahoo.com/sc/news?slug=ap-tourdefrance-doping&prov=ap&type=lgns and it talks about the Chatenay Malabry lab retesting riders from this year’s tour for the use of the new EPO drug CERA. The retesting part rang a bell. Now that Armstrong has come out of retirement, and he is now being tested to the current standards, would Chatenay-Malabry have enough stored or frozen Armstrong samples to do a comparative analysis and draw any conclusions for or against? When I watched a recent interview with Tour boss Prudhomme, although noncommital about Armstrong’s comeback, he seemed to be gleefully awaiting it. Could Tour bosses (who is the real boss anyway) be rubbing their hands for another reason and not just the publicity and the money Armstrong will generate for the tour in ’09?
Luc,
I saw the same article published on a different site. Seems rather odd, to me, that they’re just getting around to doing that kind of retesting now. I know that the lab takes a summer holiday in August, but we’re now three weeks into September and almost two months since the end of the Tour. They’re just getting around to doing these tests?
I’ve also got some misgivings about how they will use the results. It sounds as if these three (?) riders didn’t fail the initial A sample screening tests, but had “suspicious results” whatever the heck that means. I’m wondering, other than banning those riders from future ASO events (and possibly giving similar bans to any team employing said riders) whether there will be any real anti-doping cases that can be prosecuted out of these tests. It’s not clear to me whether this is a case of retesting some A samples, or whether the AFLD is doing additional work, but on the backup B samples.
Interesting, too, that they’ve finally gotten around to telling the world that Duenas’ B sample came back positive. What took so long?
And, like you, I wonder if the Tour bosses are lying in wait, hoping to snare Lance Armstrong with some positive anti-doping test result. Imagine the publicity for that. “We finally got him!” will be blaring from certain newspapers’ headlines, I’d expect. Ol’ Mistah Lance is going to have to make sure he’s 100-percent squeaky clean if he gets to race the 2009 Tour. Otherwise, the PR flaks will be having a heyday crowing about how great the new era of anti-doping tests is that it snagged the biggest fish of all. It takes some serious cojones to stage a comeback, given all that. I hope Lance knows what he’s really in for. I don’t think it’s going to be pretty.
Hi,
The comeback of Lance has opened the pandor box. There is not only FBI lab which does false or bad studies… the Coyle’s study on Lance seems to be a fake
http://www.sportsscientists.com/2008/09/coyle-and-armstrong-research-errors.html
and
http://www.sportsscientists.com/2008/09/coyle-armstrong-research-installment-2.html
About the FBI lab, I’m a bit disappointed that no lawyer discovered that that kind of analyses were not use in most of others countries (as I know). That is very different of WADA labs which are using maybe different testing but at least they share the science.
Prudhomme was just polite with the comeback of Lance, not happy. He stated clearly that Lance would have to explain his 6 EPO samples! In France the newspapers resumed his words by “Oui, mais…”. So Lance on TDF is not done!
As to Lance making a comeback, I agree Jean that he is indeed opening Pandora’s box. Lets just say, for the sake of argument here, that Floyd is indeed innocent of what he is accused and convicted of. He has since shown the impossibility of proving his innocence (to those that matter) as the system currently works. So now here comes Lance back into the fray. Fair game once again. Anybody who doesn’t belive that there are people out there who WANT to take him down must have been on another planet during those final few TDF wins, and even since his retirement.
SO…how does Sir Lance protect himself from any potential ‘junk science’ testing and/or intentional manipulation intended to reinforce what is already thought by many (he HAD to have cheated, so NOW we have a chance to prove it! We just need to make the results show what we already know). I would think that at any and all official doping control situations, he’s want to find a way to do a double sample and have that C and D sample somehow documented (so that it would stand up in a CAS appeal) and save them somewhere quite safe with a chain of custody beyond reproach. IF his A/B samples ever came up positive for ANYTHING, he could pay to have his C/D tested at at another lab (say UCLA, if they would even cooperate…being under the WADA umbrella, that might not be possible). And if he is unable to get his EXTRA samples tested at a WADA approved facility, would WADA accept any results from any other lab, no matter their crediantials? I think not. CAS has shown that they are completly able to overlook actual evidence in making a decision.
If I were Lance, I’d be VERY AFRAID to subject myself to this. He is looking to the future, but it’s his past that would truly be in jeapordy. Thats a lot to gamble for the payoff he is looking for IMO.
Jean,
No arguments from me. Lance is certainly opening a Pandora’s Box by coming back to the world of professional cycling. Whatever his true motivation, he’s going to be dogged by the stories of doping — even if he never tests positive during his comeback. Some people will take his not getting caught as evidence that he’s a genius who can make the anti-doping cops look like Inspector Clousseau. And on the other side, people will say that the fact he’s never been caught is proof positive that he’s never doped. The truth, I suspect, lies somewhere in between. It’s a huge gamble his taking, however it turns out.
I’m not so surprised that no lawyers found out that bullet lead analysis isn’t used elsewhere. None of them ever found out that the foundation of the theory had never been tested, either. Which doesn’t say much about the investigative powers they might have brought to bear for their clients’ benefit. Or, more to the point, the lack of investigative powers. Forty years before such a story broke? And some of that time was during the Woodward and Bernstein-inspired surge in investigative journalism?
I can imagine that Christian Prudhomme and the others main players at the ASO were not exactly thrilled by the prospect of Lance coming back. As you say, Prudhomme’s comments continue with the phrase, “Yes, but …” But for the Tour organization, in some ways it’s a win-win situation. Either Lance participates and creates a great PR interest in the race, which benefits ASO and the sponsors, or Lance gets caught up in a doping investigation, which draws attention to the ASO, the sponsors, and how well the fight against doping is going. I don’t really see how the Tour loses if Armstrong races there next year.
Also, I wonder how long his comeback will last. Is this a one-year deal, or is he coming back to race until he can’t turn the pedals over any more? I haven’t got a clue. Whatever the case, he’s going to inject some excitement — at least on this side of the pond — into the whole event. I suspect the same will be true in parts of Europe, too. And all of that will be good for a number of sponsors. Including such long-time Armstrong supporters as Trek and Nike.
I don’t see C/D samples as a possibility. I would think that the doping control techs in charge of collecting the samples would simply tell Lance or anyone else so inclined, to leave after obtaining the A/B samples. Then you would have the whole chain-of-custody issue to deal with. not impossible but rather difficult. Don’t expect an arbitration hearing to be as forgiving of an outsider’s procedural errors as they were of LNDD’s. And finally, any hearing could simply refuse to admit such additional test results as evidence, because there is nothing in the WADA procedures for such.
If ASO, LNDD, etc. are out to get Lance to the extent that they would manufacture a false positive, he is well and truly screwed from the get-go. If, one the other hand, the powers that be are willing to test him as fairly as any other rider and let the chips fall as they may, he has a chance.
Rantheads, I’ve been on vacation, but a few points.
First, this is Lance Armstrong we’re talking about, a guy with the power and resources to stand up to anything that the ADAs and cycling establishment can throw at him. If Landis spent $2 million on his defense. Lance could afford $20 million, or more. If Landis had to search for experts to testify for him, Lance already has connections with top experts in industry and academia. Maybe you all are right and Lance is opening Pandora’s box. But if the ADAs or the ASO decide to take on Lance, they’ll be opening a much larger box.
And note that the ADAs have never dared to take on Lance directly.
Further, if the ADAs have anything on Lance, they could have used what they have even if Lance had not announced a comeback.
Larry:
You have some good point, but we are playing a rigged game here. WADA makes, enforces, and interprets the rules. Lance has experts ready to go to bat? Fine, but remember your own post on how the battle of experts can play out. An arbitration panel can always ignore the defense experts (after all, they probably are not currently employed by a WADA lab, and hence lack the same degree of expertise as the WADA experts, so the thinking goes), or even ignore both sets of experts and decide the case on other evidence, their gut reaction, or even on whether or not the defense lawyers are properly gentlemanly in their behavior.
Remember, we are talking here about Lance in 2009, not in 1999-2005. Whatever they have or don’t have on him from the past is of no concern.
Hope you enjoyed your vacation.
Larry,
You’re right, we are talking about Lance, and not someone of lesser presence and means. He certainly has the bucks and the cojones to challenge the anti-doping system in a way that few others could. And I don’t doubt that if he had to, be could bring staggering resources to his defense. Much more than Floyd or most others could or would.
As William noted, given the way the game is currently rigged, I wonder if he would be able to prevail. At the same time, I hope he never will have to be in the position to defend himself against charges of doping. It would be a watershed moment in terms of the sport and in terms of the anti-doping agencies, should such an event occur. And perhaps it would be the thing that would bring about real change on the system. Then again, having rigged the game the way they have, I could also see an outcome where he would be found guilty, regardless of whether the evidence truly proved his guilt.
I’m hoping that the coming season will have fewer doping scandals, and that the focus of mainstream media stories about cycling will be more on the racing and less on the seemier side of the sport. I’m not willing to bet on that, however.
Let us not forget the every popular court of public opinion. If Lance were to win the TdF, there are some who would say it is impossible for someone of his age to win without chemical aid after taking several years off. Donna Torres got some of that this summer. And if he doesn’t win, especially if his is well off the pace, some will say this is evidence that his past performance was artificially boosted. Either way, the nay-sayers will not care a wit whether or not his tests are negative.
How much this will affect general public opinion, it is hard to say.
On a separate note, I see in the local paper that NASCAR will start random testing of drivers and put crews. In the past, they only tested if there was “significant evidence” of drug use.
William,
“Significant evidence”? I wonder what that would have been. I’m imagining that they would have needed to see the needle sticking out of someone’s arm in the past. 😉
Who knows exactly what the term means – I suspect whatever NASCAR deemed that it meaned.
I do wonder to what extent PED usage occurs in NASCAR racing. I can suppose that there could be possible benefits for either drivers or pit crews, both of which do undergo physical effort in competition. But then the factor of equipment (car, tires) seems to be a large amount of what determines who wins as well as drivers skill. I am not sure to what extent anyone wins because they produce a greater physical effort.
Maybe this is more a publicity thing.
I suspect you’ve hit the nail on the head there. It seems to me that a lot of the “doping” or cheating that goes on in NASCAR has more to do with making the car perform better, by bending or breaking various rules related to the car’s set-up. That said, a drug that could enhance the driver’s concentration and/or help keep the driver calm under stress might be a factor. But I wonder whether it would be as big a factor as tweaking the car’s performance would.
With all the uproar over doping these days, who wouldn’t want to jump on the good publicity bandwagon?