Today, while reading a story in the print edition of The New York Times about an odd little religious sect in Utah, I caught a glimpse of a story that has a little bit (somewhat tangentially, perhaps) to do with one of my more frequently addressed subjects — the Floyd Landis case. Many electrons have been spilled over the idea that had Landis’ case been a criminal case subject to the US courts, he might have gotten a better shake than he did in arbitration. Maybe so, maybe not. What caught my eye was the headline on the jump, which said:
Court Weighs Format for Lab Evidence
The callout in the middle of the second half of the article asks the question:
Should juries get crime reports in person or on paper?
OK, this is going to be interesting, I thought. So I went back to the start of the story (Court Weights How Juries Should Hear Lab Evidence, by Adam Liptak) and dug in. The lede paragraph and the second paragraph read:
The Supreme Court heard arguments on Monday in a case that could have broad implications for how prosecutors present evidence from crime laboratories at trial.
Many states allow reports from crime laboratories to be submitted to juries through written certifications rather than live testimony. That practice is convenient, but it may run afoul of the Sixth Amendment’s “confrontation clause,” which guarantees criminal defendants the right to confront the witnesses against them.
So, at issue is whether or not a written document with lab results is enough, or whether the lab technicians or other personnel who performed the tests must actually answer questions from defense lawyers. Makes an interesting parallel to the idea that the material contained in the lab documentation package is all that an anti-doping agency has to present in order to make their case against an athlete.
The article goes on to note that the Supreme Court justices appear to be aware that some criminal lab work has been less than perfect. More to the point, there have been some cases where the results were just plain wrong. As Liptak notes:
Justice Stephen G. Breyer cited a supporting brief filed by the National Innocence Network in the case, Melendez-Diaz v. Massachusetts, No. 07-591. The brief, Justice Breyer said, “is filled with horror stories of how police labs or other labs have really been way off base and moreover really wrong.”
“They lost the results,” Justice Breyer said of the laboratories. “They got it all wrong.”
Without the ability to cross-examine those who actually performed the tests, such information might not come out. The article goes on:
Martha Coakley, the attorney general of Massachusetts, argued that requiring live testimony from laboratory analysts “would be an undue burden with very little benefit to the defendant.”
Perhaps. Perhaps not. In Landis’ first arbitration hearing (May 2007 at Pepperdine School of Law), the testimony of the lab techs may not have been much help in the ultimate decision, but it painted a picture of how the lab work was performed that wasn’t all that complementary. In a real court, perhaps that would have been enough to cast the interpretation of their work in doubt, which might have led to an entirely different outcome — Landis being found not guilty of the charges against him. Their testimony certainly raised questions about how well the technicians performed their jobs and how accurate the results were.
Ms. Coakley added that the results of analysis conducted by machines should not be subject to the confrontation clause. The tests, she said, produce “100 percent accuracy by and large.”
But she conceded that there was a human element in the process, as the results produced by machines must be interpreted. “A chemist, if properly trained,” she said, “can say by the separation of the chemicals that these three or four or whatever the elements are equal cocaine.”
I think, over the course of the debate about what Landis’ test results mean, that most people would concede that test results are only as good as the people performing the tests and interpreting them. And that there are situations where the results are far less than 100% reliable. What she says would only be true if the tests were performed exactly right, and the operational conditions were exactly right. If a sample were contaminated with another substance, for example, that could throw the results and interpretation of those results into doubt.
At least one of the justices sees a distinction between various types of laboratory tests.
Some justices suggested that chemical analysis was quite reliable and might be distinguished from other laboratory work.
“Now, if there are new tests, complex DNA tests and so forth, I suppose there is a lot to ask about,” said Justice Anthony M. Kennedy. “Standard blood alcohol, not much to ask about.”
Good point, that. Some tests are pretty straightforward. Some aren’t. It sounds to me like Justice Kennedy is right on the mark. More complicated tests could require more complicated questioning to determine what the results really mean (and whether those results are actually valid).
Then again:
Jeffrey L. Fisher, representing Mr. Melendez-Diaz, disputed that distinction on practical grounds, saying that defendants might want to know quite a bit about the chemical analysis of substances determined to be illegal drugs. “What test was performed?” Mr. Fisher asked, giving examples. “What’s the error rate on that test? How do your protocols work? What are your experience and credentials in analyzing those?”
Students of the Landis case, don’t some of those questions sound familiar? Without knowing the answers to some of these questions, how do we really know what a test’s results mean?
Mr. Fisher also objected to the distinction as a matter of principle. If the confrontation clause applied, he said, the reliability of the evidence would not matter and the defendant could put the government to the burden of proving its case through live testimony.
Sounds much like what Landis’ defense team was trying to do, dosen’t it? Except that, in his case, he had to try and “flip” the burden of proof back onto USADA, rather than force USADA to prove their case. The nature of the anti-doping system being what it is, by the time his case hit arbitration, he was already presumed guilty.
Some jurisdictions, it seems, already follow the procedure being discussed at the Supreme Court. If I’m following Liptak’s story correctly, one state that does is California, Landis’ home. And another place that does is the District of Columbia. Apparently, the number of Drug Enforcement Agency lab chemists having to testify in court increased by 5 times after the DC courts refused to accept analysts’ certificates as unquestionable proof.
Which brings me to my point. While much was said (including by this author) of how a regular court proceeding would be better in Landis’ case, this story makes me wonder whether or not it would have made a difference. I think so, if the assertion that California allows for the testimony of lab personnel is correct. At the original arbitration hearings in Malibu, Mme. Frelat and Mme. Mongongu were anything but effective witnesses in establishing the quality of work they performed. But if the standard were different, and merely the presentation of lab results were required to prove the presence of banned substances, I don’t know that anything would have changed.
Which is to say, “Be careful what you wish for …”
Any test, no matter how simple, is subject to error. I know this all too well: when I was drafted, the Army, in its infinite wisdom, determined that my blood type was A positive, when both before and after my hitch, I was O positive. Now, since I don’t think it is possible for someone to change blood types (twice!), obviously the Army screwed up somehow. Maybe something was done wrong in conducting the test, maybe there was contamination, or perhaps it was just a clerical error in recording test results. Fortunately, I never needed a transfusion.
Now, the test for blood type is pretty simple, I have done it myself in HS biology. You have an A and a B antibody, and you take 2 small blood samples and test each sample with on of the antibodies. One, the other, both, or neither will visibly react and that tells you what blood type you have.
If this was a court case where part of the evidence against me was that A type blood was found at the crime scene, you’d better bet I’d want those lab techs to testify in court so I could find out what happened.
Good point. Even the simpler tests are only as good as the people doing them.
Part of the problem here is the desire for “scientific certainty”. We have a reverence for science and scientific truth. We’re willing to recognize that labs can screw up, but we still want to believe that most debates can be resolved by the competent application of the right science.
Truth is, some of the science in these cases is flawed.
For me, the most interesting part of the Landis case is the limitations inherent in the science of anti-doping. At their best, the tests used to convict Landis are not all that good, and the science behind a lot of what the ADAs are doing is not that good, either.
Part of the problem is that the tests are indirect — they are trying to measure something that, arguably, is associated with doping. For Landis, the measurement was a tiny difference in the atomic structure of the carbon contained in two substances found in Landis’ urine. Even if this measurement was performed properly (and supposedly, it’s a difficult measurement to perform), the bigger question is “so what?” Who says that the measurement indicates that Landis doped? Who says that the measured difference could not have been produced naturally?
In other words, good science requires not just valid observation and measurement, but also valid understanding of the meaning of what the scientists have observed and measured. USADA’s case against Landis came down to an argument that there was only one possible meaning of the Landis test results — he must have doped. But such a conclusion goes beyond the science. AT BEST the science can say that the Landis test results are highly unusual for a person who was not doping, but usual for persons who dope.
Over at TBV, we’ve had occasion to carefully examine these assumptions, and there are strong reasons to believe that the Landis test results may not BE all that unusual. But putting this examination to one side, it still comes down to a calculation of the odds, and ultimately to a characterization of these odds: assuming that the tests were performed correctly, then the Landis results were a bit unusual, or somewhat unusual, or highly unusual, or damn unusual, or a freak occurrence of nature, if Landis was not doping. You can take your choice of any of these characterizations — the point is that at the end of the day, even if the tests were performed properly, Landis cannot be shown to have doped as a matter of scientific fact. At the end of the day, we’re talking about whether the scientists proved the LIKELIHOOD that Landis doped to a degree of probability that we find to be satisfactory under all of the circumstances. It’s a question of what odds we find to be satisfactory, what chance of error we believe to be sufficiently remote.
At the end of the day, it’s not a question of science. At the end of the day, it’s a human determination of what is fair and what is just. We can use the science to guide our determination, but we cannot hide behind the science. The responsibility for making the right determination is ours.
Larry,
Very well put, as usual. The only thing I’d add to what you said would be that justice is best served when those making a decision are given a basis on which to decide. With scientific evidence, that means testimony from experts on what the theory and practice of the particular science is, and what the results mean. You’ve done a great job of pointing out that it’s not always so black and white. When only test results on paper are allowed, I’m afraid that such black and white distinctions are easier to draw, unless we know more about how the tests were performed, how they were supposed to be performed, the theory behind the testing and what the data really means.
Rant –
You write that justice is best served when the decision is based on all of the available evidence, and of course this is true. But oddly, there WAS a great deal of information available to the decision-makers in the Landis case. The scientific evidence WAS closely examined in two arbitration hearings (one of them public) by some of the leading experts in the world. The science behind these tests was thoroughly discussed and carefully explained, in terms that allowed idiots like me — with a great deal of help from others here and at TBV — to come to a decent understanding of the meaning of the science.
There is a cost to this kind of examination — in the Landis case, the cost was several million dollars. If we think the problem is that the Landis case required a more thorough examination of the evidence, then we’d need to be prepared to pay a greater cost. Of course, we cannot afford to devote such extensive resources to every case that turns on scientific evidence — every drunk driving conviction, for example. It’s uncomfortable, but our justice system must be run on a budget, and we have to balance costs against benefits.
I will also tell you that no lawyer in the history of mankind, or at least no lawyer who ever lost a case, thought that he or she was given sufficient access to the facts, or a sufficient opportunity to argue the case. This is in the nature of things.
The Landis case poses a problem that does NOT exist in every case: the case turned on an examination of evidence (Landis’ urine samples) that did not exist once the case went to arbitration. The Landis defense team was allowed to examine the test results, but they could not examine the urine itself. Unfortunately, this limitation is inherent in drug testing, and in some other cases (like breath tests for drunk drivers).
With the primary evidence unavailable, the Landis defense was left to examine the results of the testing of the primary evidence. We might all wish that we knew more about how this testing was performed. But at the end of the day, you’re never going to have conclusive, objective evidence that a test was performed properly. It’s the old “proving a negative” problem that Landis and others have pointed out: it’s impossible to prove that an athlete did not dope, as the athlete’s activities cannot be monitored and documented 24 hours a day. In similar fashion, a lab can provide records of how it performed a test, but it cannot prove with certainty that the tests weren’t botched in a way that was never recorded.
What I’m trying to say is, the Landis case (and nearly every other case) turns on the human and subjective process of judging the available evidence — even if the evidence is scientific evidence, and even if the evidence is not as complete as we might like. The human and subjective nature of this process makes us all uncomfortable, as we know that humans make mistakes. If it were possible, we’d love to replace this process with one based on hard, cold, objective science, where evidence could be evaluated by machines instead of human beings, where uncertainty would not exist and where judgment would be mechanical and beyond question. But no such scientific process exists, and when we pretend that such processes DO exist, we deny the responsibility we have as human beings to exercise our imperfect human judgment with due care and responsibility.
This is, I think, where our criticism of the Landis decision should be focused — not on some hope that the science can save us from the problems inherent in human judgment, but in an examination of whether the authorities reached the proper decision based on the available evidence.
One thing to consider is how the “scientific” evidence correlates with other evidence. In a DUI case, for example, the arresting officer can testify to the fact that the defendant was driving erratically, staggered around and had slurred speech, etc. to back up any blood or breath test administered. In fact, here in Missouri, it is legally possible to get a DUI conviction even if the test results are below .08% BAC (although I suspect it would be rather hard to get such a conviction in practice). Some jurisdictions video tape DUI arrests to provide such corroboration of any testing.
I suspect that in many criminal cases where say, DNA testing is used, there is also other evidence to indicate to some degree that the defendant is in fact the perpetrator: witness accounts, fingerprints, etc.
In the Landis case, at least as argued here and at TBV, many on the anti-Landis side argued that the S17 comeback was at least highly unlikely unless Landis was on something, while those on the pro-Landis side pointed to figures indicating that his wattage was within those he obtained in training. This indicates a big problem in attempting to use such “evidence”: it is largely a matter of opinion as to whether or not a given performance would be possible (for a given athlete) without the use of some form of dope.
What other kind of evidence could possibly be used? The presence of any degree of Omerta, coupled with the probability that much actually administration of dope is done in private, means that it is unlikely that much is to be gained by attempting to get other riders to drop a dime on someone. Some forms of doping would probably require the assistance some type of medically trained personnel who might be possibly turned, but there are forms of doping which could be self-administered, at least if the dopee was willing to take risks with dosage etc. Then there are the suppliers.
Larry,
Understood. I think what’s being argued in the Supreme Court case is just the kind of “scientific certainty” that you point out does not really exist. We’d like for the science to be perfect, and for it to settle questions objectively, but ultimately, there’s someone looking at the data and making a judgment. Whether that’s a lab technician or a judge or arbitration panel, someone will take in the data and come to a conclusion. We can argue whether the conclusion is justified. I don’t see a system where there is 100% certainty, even if that is the goal.
What I find interesting about the case at the Supreme Court is that a similar argument is being made as in the Landis case. That being, the lab results give all the relevant information, and no testimony is needed from the lab techs about what procedures they used, how they performed them, etc. That would be similar to the WADA/USADA/LNDD argument that all of the information needed to decide a case is in the lab documentation package.
If the documentation were perfect (never happens), perhaps this might even be the case. But since humans are fallible, mistakes can be made, and those mistakes can have a real impact on others. Imagine receiving the news you’d tested positive for some awful disease. You’d want to know more about the test, how it’s performed, what the science of it is, and so forth, right? And you’d probably want to take that information and run it by another expert, to see if he/she agrees.
In the Landis case, much information was available, though USADA tried (and in some cases succeeded) in blocking his access to various bits of information. Under the constraints imposed by arbitration, his lawyers did the best they could, but perhaps they didn’t have enough time or access to the right information. I’m sure that there are many defense lawyers who would say the same thing when a result goes against their clients.
Compared to a criminal trial, where the defense puts on their case under somewhat fewer time constraints (or such is my impression), I think that the arbitration system’s focus on giving each side equal time is less favorable to the defense. It sounds reasonable on the surface, but given the hurdles that the athlete must clear in order to win against the ADAs, it seems as though despite the “equal time” provision, the athlete is still at a disadvantage.
One thing the Landis case has done is bring a number of issues regarding anti-doping testing, enforcement and adjudication to the light of day. We may not like the outcome of his case, but by knowing how things are done, we’re able to agitate for change. Were it not for the rule that allowed for an open hearing, and were it not for his lawyers ability to question the lab techs and others during the Pepperdine hearing, there is much we wouldn’t know about the world of anti-doping. I’d say it’s a good thing to allow for such testimony, and to allow for public hearings.
When we rely only on written documents and only written documents, we rely on the accuracy of those documents. I see a danger in that. If the documents are perfect, and everything was done perfectly, perhaps that would be OK. But that requires a leap of faith, in my opinion.
Imperfect as humans are, I see a need both for the ability to question those who perform the tests, and those who design/develop the tests, so that whoever decides the case can be aware of any issues (like sloppy lab technique, sloppy documentation, lack of verification that tests actually do what they are supposed to do) before rendering a judgment. When the judgment is made, we may disagree with those who made it. But if a reasonably thorough presentation and discussion of the evidence is made, the chances of a fair decision are greater.
So, in the end, I agree with you. The process of having humans make these judgment calls, while not perfect, is a pretty good one. While we’d like for science and scientific arguments to give us definitive answers, it will be a long time (if ever) before things have evolved to the point where we can have the kind of confidence that I’d want in order to accept scientific evidence as proof beyond reproach. And it’s for that reason that I’m uncomfortable with the idea of allowing unquestioned scientific evidence as the basis for a judgment against anyone, whether that’s Floyd Landis or another athlete accused of doping, or a person being tried in a criminal court.
Sometimes we just have to accept the ugly truth. We cannot stop or change for the better a wrong that has occured.
Dirty playing by supposed “righteous” public entities is certainly not a thing invented yesterday.
It seems to me we are starting to miss the point.
Somewhere along the line – the woes of an athlete accused and convicted first by “public lynching” then by a “stacked” book of rules and regulations and a governing structure that is stacked against actually finding out the TRUTH, is left in the twilight.
So we are left discuss symantics and the finer understanding of the law or the limitations of our scientific abilities. Nothing is wrong of course with such discussions. As long as we do not forget that a wrong has been done. Perhaps this is what IS important – TO REMEMBER! That someone will REMEMBER, that a wrong was done!
Take the story of Major Taylor and that bigotted McFarland. There is a book by Todd Balff – titled “Major” – It is not particularly written as a “page turner” in mind – but one certainly gets a great deal of information about the very beginnings of “procycling” as we have come to know it today.
Perhaps after reading this story one can come away with a surer sense that this “present day” situation in the cycling world has been going on for a very long time. And also that the “real Question” is not about honesty or a case of cheating. Rather a realization that our world is populated by people who take joy in tearing others down.
That the issues of poor sportsmanship is nothing “modern.” That “cheating” was not recently created by todays crop of “pro cyclists” and “handlers.” That “cheating and unfair play” is rampant throughout the sport of cycling. And it will be a “part of cycling” no matter what we try to do.
It is my personal opinion that trying to regulate the morals of individuals never works. You may write all the laws you wish – but you will NEVER control the moral behavior of selfcentered ego motivated beings.
So I suppose if I “wish for anything” it is that some REMEMBER – because somethings need the passage of time to heal the wrongs that are done.
Cheating is not limited to pro cycling, not is cheating only confined to the realm of chemical enhancement of abilities. Athletes have been bribed to “throw” events, I am sure even before the Chicago “Black Sox”, “ringers” have been used. Depending on the sport, athletes have been known to develop any number of dirty tricks to gain an advantage while escaping notice of officials. In high school and college sports, academic fraud in various forms has been committed in order to get athletes eligible (I have been subject to various subtle and not so subtle attempts during my career as a teacher). High schools recruit players to move into the district against rules; colleges commit all sorts of violations regarding recruiting in order to get top athletes. Even local “little leagues” in any number of sports have people doing various and sundry things in order to “stack” a team. I could go on and on.
Some of these things can be pretty easy to prove, if you get the right evidence. A teacher shows he has a player getting an “F” in his grade book, but the school record shows a “C”, you know something funny went on. Somebody signs a kid up on an under 12 team, and shows a birth certificate copy to seemingly verify the age, but a check with the Bureau of Vital Statistics indicates the BC copy was altered or forged: pretty clear. But others can be unclear. When I lived in Texas, there were several cases in the news where a HS student’s family moved into a new school district. According to Texas rules (and maybe in other states too), if a family moves for job related reasons, the athlete can play at the new school, but if the authorities deem the move was made for athletic reasons, the student can’t play (maybe for a year? I forget the exact details). This could be rather a judgement call at times, maybe Dad looks for and gets a job in the new district in order for junior to attend a school with a top team: how do you rule that?
Anyway, does this mean we should forget about trying to control all these things? Laissez-faire to the max? Or should we make some effort to control things, knowing we may not succeed 100% of the time? I think we need to make some effort, but let’s be sure also that we don’t needlessly penalize athletes with faulty data.
William,
I do not think sports or for that matter most things that involve people interacting for pay, prize money, earning etc, should be “Laissez-faire to the max?” But I absolutely believe that it is a constant struggle for society to find itself.
I absolutely concure with you – we must make the effort, we have to fight the battles, but in truth it appears to me as sometimes the only thing we have at the end to actually hold on to is – TO REMEMBER – that the wrong was done.
It is always a battle when idiologues are running the show. If they are deeply entrenched then the fight for right seems always a losing battle. But the battle must be faught, as you say.
As some here and at other places keep putting forth – “Why do the pro racers not fight for themselves, for their rights, for fair play?” Let’s face it this is much easier said then done. How many of us would freely and without a worry in the world jeperdize our ability to make an income? Why should a pro cyclist be any different?
For instance – can you imagine how many people were involved as “workers” in the present day fiasco that is turning our financial security into a circus of horrors? Why did no one seem to “stand up and take a stand?”
You cannot tell me that there was no one who did not know that giving “bad credit loans” to individuals who could have only the hairs breadth of a chance to pay it off, came as a great surprise to the “lenders”?
The average “consumer” is not the shapest tack in the world. What they are is no different then what we all can be held accountable for – we are greedy. We all want to “live better.”
We can give as much “lip-service” as we want to being upstanding citizens, but if our secret motivation is to feed that hungry ghost that never gets full then we have to accept that the question of “morality” is nothing more then a “shadow play” to supply the illusion of moral ground.
Question: Does a defendant in a criminal trial get access to the entire lab documentation? In other words, does he have access to all the information, but cannot cross examine it, or does he only have access to the certification?
Landis had a two part problem with the lab materials: He had limited access to the lab personnel. More importantly, the lab was not required to send him the entirety of the evidence supporting the claim.
Is that the problem in the criminal system as well? That would be surprising (disappointing) to me. This would also create a different list of questions.
Michael,
I see the potential for the same kinds of problems there, but with a more robust discovery process, I suspect that a defendant in a criminal trial is more likely to get the entirety of the documentation, rather than the limited amount that Landis was given access to. But I would also expect that prosecution lawyers and lawyers for the lab involved might try to oppose such efforts.
Perhaps one of our legal-eagle readers can enlighten us.