In science, for results to be considered valid you should be able to repeat an experiment or an analysis of data and come up with the same result every time. Without repeatability, you haven’t proven anything.
Sometimes, however, it’s not always practical or affordable to repeat an experiment. For example, physicists looking for the elusive interaction between neutrinos and other particles may have only so much money to spend on shooting a particle beam through the earth at some distant target, with the hopes that a few interactions might be recorded. (Or, in Floyd Landis’ case, there may be too little of his B sample left to adequately perform further tests.)
Once the experiment is over, there would be a mountain of data to analyze. And in the analysis, perhaps they would find those elusive events. Hopefully, other groups would repeat the experiment in other locations and achieve the same result. But if not, one can always re-analyze the data to see if the original interpretation is correct.
With the Landis case, LNDD provided a number of mass spectrometry graphs in the lab documentation pack that supposedly prove the presence of synthetic testosterone. As Dr. de Boer, Floyd Landis’ observer, noted in his comments after the B sample tests were run, the data provided didn’t, at least in his eyes, meet WADA’s standards of proof. One of those standards requires the raw data be presented, along with the graphs. It’s not there.
There are a number of reasons why the raw data is important, as TBV and Duckstrap pointed out in their article, Where are the data files?
Regardless of what you believe about the truth of the accusations against Landis, these data files are very important. First, because WADA requires their presence to back up a positive finding. But equally importantly, to allow for re-analysis to see if the data has been properly interpreted and understood.
With raw data in hand, USADA (for example) could run that data through the appropriate software and see if their experts came to the same conclusion as LNDD. For an organization that claims to be seeking the truth, this would be an important step before bringing anti-doping charges against any athlete. Suppose on re-analysis using another GC/MS software package they found different results than LNDD. Would they pursue a case, or would they go back to the lab and say, “Our re-analysis doesn’t agree, therefore we are not going to pursue this matter.”
And Team Landis should have that data, as it is crucial to their defense to see whether, upon re-analysis they get the same results as LNDD. Perhaps, with more sophisticated, more modern software either group doing such a re-analysis would find that the data led to different conclusions than LNDD reached. Or perhaps the data would agree.
But if it can’t be repeated, then there is something wrong. Bad data, bad analysis at LNDD, or … ? And without that verification, how are we supposed to believe what’s been put forth? Just because LNDD says so? That’s not science, that’s faith. And some people have faith in LNDD and some don’t. But either way, if the original interpretation is correct, it should stand up on re-analysis. If it’s not correct, that will come out upon re-analysis as well.
Since it’s been almost 8 months since the B sample tests were conducted, the credibility and reliability of any data that surfaces now may be suspect. It should have been produced at the very beginning, and it’s disturbing that it wasn’t. Was this sloppiness on the part of the lab? Or was it an attempt by one agency or another to push the case forward without following the rules?
Has LNDD already re-analyzed their data and seen that it’s wrong? If they have, are they keeping quiet because of the embarrassment they will suffer should that come out in public? Have they destroyed the data? Have they manipulated the data in some way to “verify” their results? Does USADA have the raw data? Have they analyzed it? If so, why haven’t they provided the information to Landis and his defense team?
All of these questions come about because the lab didn’t meet WADA’s requirements to provide the raw data in the first place. But, as “just bitch slap me please” noted in his comment on yesterday’s post:
If I was USADA and in the drivers seat with strong data, I would be happy to lay it on the table for all to see, weep, and rend garments over.
Indeed, I can’t think of why they wouldn’t put it out there. They’re the ones pressing the case against Landis. If they’ve got a slam-dunk case, why hold back? On the other hand, there’s a part of me that suspects the reason USADA and LNDD are holding back is because they know the raw data doesn’t back them up. And while they may eventually provide it to Team Landis, it may be at 8:59 a.m. on May 14th, just before the hearings begin. And just late enough to make any re-analysis difficult for the Landis defense.
This is exactly what I was thinking…if its good, why is it hidden…if its hidden, why isn’t it good?
Someone needs to pull the plug on this one before they face serious embarrassment, sanctions and loss of jobs. Is LNDD keeping quiet because they’re under the same ‘rules’ as the ADA’s not to talk about current cases. Or are they trying hopelessly to cover their asses.
This is looking more and more like the proverbial ‘sacrifice the one to save the many’ that most villains in movies try to rationalize through the evil monologue…are all these dudes wearing eye patches, laughing slowly and sitting in huge villain chairs… It’s time for this thing to be over so we can go back to talking about world peace or something…muuhaahaahaaaa
Rant, if the data is required by WADA rule and the data is not there, could this be a procedural “le dunk de la mort”? Even if the data shows up at the 11th hour, the undeniable fact is that it was not provided with the lab package (right?) nor was it in the USADA arbitration submission (right?).
James,
I like the looks of those villain chairs. Those should be a must-have item on Dr. Evil’s Christmas list. 😉
Steve,
If the data appears at the last possible moment, I don’t know if it would qualify as “le dunk de la mort.” It’s certainly a procedural error, and it could be. But my guess for a dunk of death would be something much stronger. A violation that there’s no way to get around, even with an attempted last-minute delivery of required data.
– Rant
Has Floyd requested the raw data from the arbiters? Does Floyd win if the data never shows up?
Will,
My understanding is yes to your first question, and I don’t know to the second. The data is required by WADA’s rules, but whether or not the arbiters would find that the missing data would be enough to toss the case is hard to say. I would like it (and it would be the right thing in my mind) if they did, but it’s not guaranteed.
An aside — do you know if Floyd intends to show up at the Tour de Georgia for fund raising or any other purpose? His site doesn’t mention it. If you don’t know, do you know where I can find out or where he’s blogging these days?
Will,
I haven’t heard anything about whether Floyd will show up at the Tour de Georgia. If he does, I’ll post an announcement as soon as I find out, and I’m sure you’ll be able to find links at TBV and at the Floyd Fairness Fund website (http://www.floydfairnessfund.org). I haven’t seen much that he’s blogging these days, or writing in the various discussion forums. But there’s an update at his website today about how his hip recovery is going. With only 6 weeks to go until the hearing, I imagine that things are going to get even busier. As soon as I hear anything, I’ll let you know.
It seems that the standards for accepting a positive result in doping tests are extrememly ad-hoc and unstandardized. As a scientist, I can say that if someone submitted the analysis of Landis’s blood as a research paper to a peer-reviewed journal it would be rejected. As in the presumption of innocence in a legal case, in statistics there is the familiar null-hypothesis. You work under the assumption that the data are produced by chance, and it takes very controlled conditions to have confidence that data are as extreme as they are, or more extreme, assuming the null hypothesis. The relevant variables and factors in doping tests seem to be almost intractable, thus it seems the labs can only infer likely sources of unusual levels of substances in the athletes’ blood. And given the extreme physiological differences between professional athletes and the normal population, it must be nearly impossible to define what ‘unusual’ levels are. Having only reviewed the Landis case, it seems impossible to rule out other explanations for his testosterone levels in statistically sound fashion. How do they answer the basic question of ‘how probable is that Floyd Landis injected synthetic testosterone’? I am not familiar with the labs that do the testing, nor with their standards for data analysis, but I am certain that these standards would not meet the minimal standards of the professional scientific community. This is extremely sad because the results of these tests can destroy a person’s life, whereas often in science false results merely lead to new expteriments or a reexamination of the data in question. And I agree with the author here that it is crucial that original data be made available to independent labs to verify or refute any claims made by the prosecution.