Being an early adopter of any technology can come with some headaches. Right now, some bureaucrat hunkered down inside the walls of LNDD is probably popping an aspirin or two for all the headaches he’s getting from being one of the first to install the IsoPrime mass spectrometer.
Over at Trust But Verify is new information about the mass spectrometer that LNDD, France’s national anti-doping laboratory, used to analyze Floyd Landis’ sample from Stage 17 of the 2006 Tour:
[S]ources tell us the following:
- LNDD is using a GV Isoprime
- To their credit, it is one of the very first.
- So early, in fact, it came with Isochrom firmware and documentation.
- That has never been updated to Isoprime software.
- There are alleged to be problems with the Isochrom firmware used in an Isoprime, but the details and impact are as yet unknown.
As TBV says, it’s to their credit that they wanted to install and run a newer piece of equipment. But sometimes it’s just not good to be on the bleeding edge of a technology wave. I’ve had my own experiences with that, both personally and professionally, and these days I’m content to wait until a technology has had time to mature before plunking any of my hard-earned cash down. (Let’s just say this, it will be a long, long, long, long time before I ever install a new operating system on the day it’s released to the public. Once bitten, twice shy.)
In doing some research, the first mention of the IsoPrime in use that I can find dates to the 1998 Winter Olympics in Nagano, Japan. (See slide #32 of this presentation, given in 2002.) So if LNDD was an early adopter, they must have gotten their machine sometime in late 1997 or early 1998 would be my guess. Perhaps I’m missing something on GV’s web site, but I don’t find any mention of how long the product has been available.
Being an early adopter, at least, might explain how it is that the lab didn’t have the right documentation. It would be interesting to see what differences there are between the instructions for the IsoChrom and the IsoPrime and whether that would have any impact on how the instrument was used and the results one would achieve.
For example, would the pressure reading that Dr. Baker cited be within spec for an IsoChrom mass spectrometer?
I’m not sure if or how the software, itself, might affect the running of the instrument (or the data gathered). But once the intended software was available, and once the documentation was available, why didn’t LNDD update the software and documentation. Was that a mistake on the lab’s part, or did the manufacturer fail to send the materials? At some point, shouldn’t someone at the lab have inquired about these things?
By my guess, it sounds like they’ve had 9 or more years to get the documentation and software designed for that particular machine. How could it be that no one followed through in all that time? Wouldn’t they want to have the documentation, if for no other reason than to make sure they were maintaining and using the equipment properly? Didn’t any of the auditors for their accreditations ever request a look to verify that they had the material?
But it’s TBV’s last point that’s most intriguing. Problems with old firmware in the new equipment? Do tell. If they have any impact on data measurement, calculation or analysis, this could be a huge problem. Is this on the order of the floating point error in the original Pentium processors, or is this something on a smaller scale? Why didn’t the manufacturer update the equipment? Or did they send the parts and the parts weren’t installed?
TBV’s information about the mass spectrometer is tantalizing. I’m certainly looking forward to hearing more.
In the meantime, I wonder if the decision-makers at LNDD will be a bit more cautious the next time a newer testing instrument or a newer technology enters the market. If I were in their shoes right now, I’m pretty sure I would.
Sorry but I don’t buy the argument that the new car isn’t delivering like the old clunker. A new shiny piece of lab equipment gets all the love compared to the old MS that collects dust in the corner. Pin your blame on the people not the machine. Hell if the machine could do the test WITHOUT human intervention I might believe the data even if it was French machine.
JBSMP,
Ultimately, it goes back to people. Someone either didn’t get the software, the updates and the documentation or someone forgot to send them. But after a certain point, regardless of who that someone was, the lab bosses are responsible for whether they have the proper equipment, updates (firmware, software, etc.), and so forth. If there were problems with the machinery, someone should have known and corrected them. Machines only do what they’re programmed to do. It’s up to people to make sure the machine’s instructions are correct and the results are correct. One way or another, it all comes down to people and how well (or poorly) they do their jobs.
– Rant
The problem here is that we don’t know what, if anything, was wrong with the original soft/firmware. If there is no analytical difference, there is nothing wrong with running the old stuff forever. A wise organization, drilled in the rigidity of configuration management might very well prefer to run the old, proven stuff until it is shown to be broken.
The issue is whether the updates do affect the results in a way that makes the old versions “broken”.
It’s a tricky thing, because release notes for versions may not want to go into too much detail about what affect a problem may have had. You get things like, “bug 76532, under certain circumstances an off-by-one error happened in the kerfluffle”,
which doesn’t tell you that it chronically mis-integrated the area in the 3rd or 4th standard deviation from the peak, because the off by-one was the 3 or 4 that limited the width.
If there is any fire behind the smoke, it seems unlikely to me to turn up in easily available documentation.
Until then, I’ll place my bets on the “all metabolites” vs “one positive with the 0.8 considered” arguments as being the critical ones. Most of the rest of this seems like it’s irritating my eyes nose and throat. There may be a fire, but I don’t see it yet.
TBV
When I first read the title I thought you were going to write about the perils of professional cycling becoming an early adopter of WADA.
Encarta provides the following legal definition of negligence: “LAW a civil wrong (tort) causing injury or harm to another person or to property as the result of doing something or failing to provide a proper or reasonable level of care.” Can anyone imagine:
1. Running an incorrect version of soft/firmware on their PC?
2. Not applying a service pack to their operating system for 9 years?
3. Running complex software without manufacturer’s documentation?
Now, forget about your PC and imagine you are in a hospital, relying on diagnostic equipment to save your life (or livelihood). Your doctor says, “Well, we’re operating our new equipment with software and firmware designed for the prior generation of hardware, it’s been 9 years since we’ve updated our configuration and, to be completely forthright, we haven’t even read or seen the operating instructions. But, since this is the biggest decision in your life, we recommend “¦.” Would you consider that “reasonable care”?
I respect TBV (and read his site nearly every day along with Rant). But, I disagree that anything further need be proven by Floyd or his team. What procedures and how much money would be required to prove there is a specific bug or specific failure that would make Floyd’s results incorrect? The criminal negligence of LNDD is obvious (and we haven’t even gotten to having somebody explain why, without documentation, LNDD operated the machine outside of the manufacturers recommended ranges — i.e., at too high a pressure). We all know enough — right now — to decide this case in Floyd’s favor. At least, that’s what I think. And, oh yea, I’m running Vista/Office 12 and I have the documentation under a plant.
Will,
That’s a whole other rant. Thanks for the idea! 😉
Steve,
If this whole case were being decided in a court of law (if only), I think you’re correct, that there would be enough to decide in Floyd’s favor right now. (And I certainly wouldn’t want to be treated in a hospital being run in the manner you describe.)
Unfortunately, the system is an arbitration model, which has always struck me as odd, because it acts in a quasi-judicial manner based on a civil disputes resolution approach. But that’s what we’re stuck with at the moment. Any changes to the system will happen after Floyd’s case has come and gone.
Because of the roostered approach to justice, Floyd is stuck having to prove his innocence, with the deck stacked against him. He needs to find the strongest cards to play, and I suspect he’s holding back quite a few. By the time the whole case is layed out, my guess is that Team Landis will have a long laundry list of problems with the case.
As to which arguments are stronger, I don’t know. I do think that the metabolite/metabolites discussion is a strong line of attack, as is the uncertainty of the results.
What’s going on in the discussion of the mass spectrometer and whether it was operated correctly, etc. may be be important. But TBV raises a good point: What if we think there’s a problem there, but it turns out that it’s not as much as we think? Floyd still needs to prove his case, and he needs the strongest evidence.
So we’re back to the question of which evidence is stronger. Hard to say. And there’s a whole range of rants I could write …
Rant, you make a good point about the roostered justice system. And, I agree it is up to Floyd to prove his innocence.
Yet, negligence should be a pretty strong defense. I am not sure about US arbitration — but, I have been involved in a software-related arbitration in Sweden (of all places) where negligence was a defense that eventually settled the case (in our favor — phew!) prior to arbitration — so, I thought a further comment might be amusing.
The thing that struck me as a parallel in our case and Floyd’s was the finding of a “smoking gun”. In our case, it was illegal modifications to packaged software. There was no doubt as we found both time-stamped changes in system logs by un-authorized coders as well as correspondence authorizing the changes in contravention of contract.
In Floyd’s case, I admit there seems less obvious doubt about LNDD’s negligence regarding the operation of the spectrometer. There is no contract between the parties requiring the spectrometer to be operated in a specific way. And, TBV could be right, maybe the way LNDD operated the equipment doesn’t make any difference. But, how would anyone ever know if back leveled / incorrect software produced incorrect results in Floyd’s case? The proof would seem to require both evidence and procedures that are complex and may be impossible to execute. What we do know is that the key piece of diagnostic equipment was operated by LNDD both incorrectly (pressure) and without “reasonable care” (OK, a little judgment is required to reach the not “reasonable care” conclusion, hence my analogy).
Anyway, back to my experience with negligence and arbitration, what happened to us was that once the other side saw the negligence and it’s proof (BTW, we also took a long time — nearly 2 years and 7 rounds of complaint / response cycles to expose the argument and the supporting facts) they caved pretty quickly. While I never spoke to them, my guess was that they and their counsel felt most arbitrators have a pretty open-and-shut view of negligence — once it’s clear, it is so damning that it’s virtually impossible to counter.
I’m not a lawyer, an arbitrator or a scientist. I suppose that’s what makes Floyd’s case so compelling — from the beginning the story just didn’t seem right to a “lay person fan”. We’ve have all had to sift through lots of data, listen to many arguments (and rants!) and then apply our personal judgment. Some of the most interesting debate (to me) has been about “what is right” (i.e., regardless of the legal interpretation, should the standard be metabolite or metabolites). I guess that’s why I made the comment, with all respect to TBV, that prooving the impact of LNDD’s negligence seems both impossible and unnecessary — if the lab can’t run its diagnostic equipment in a non-controversial way, we (and the arbitrators) should have no reason to believe their results — and poor Floyd should have no reason to produce or pay for any more arguments.
But, as you point out, USADA / WADA and cycling have a roostered system of justice. I hope they are reading your site!
Some interesting reading on the GV web site.
http://www.gvinstruments.co.uk/News.asp
Don’t know the MS space or this firm. One observation comes to mind. Is there French documentation available?
The firm has a French HQ in Lyons and seems to have a big global footprint so they should have localization under control. If not could go to explaining some of the situation with the proper operation of the technology.