Curtis Meinert

Quotes

Premature communications of results:

The one lesson it seemed to me at least I learned from the UGDP is: “Publish first, Present later.” One of the frustrations that I referred to is that you’ve got to sort of keep preaching that lesson all the time. And every time you preach it people think you’re absolutely crazy. The best way to do this they say is to present first and publish later. But you know the whole story with regard to the UGDP, it seems to me. . . anything that causes this much trouble can’t be good. . . . We had lost, so to speak, before we came out of the gate in UGDP. But it’s interesting, I’ve had bloody fights, for example, . . pushing the philosophy, that is, to publish first and present later. And, you know, the AIDS people say, “That’s crazy. The way to communicate is – you’ve got to present this stuff right up front and then publish it later.” So I think that’s the issue. . . Furthermore, I had a student that did some work on this and discovers that if you present first and publish later, you’re less likely to ever publish. You know, there’s lots of reasons. I don’t know. Lessons. (7)

I always feel as though I’ve sort of like have a prison record because I was involved in the UGDP. I’m proud of it, by the way.

[Badge of honor for us all.] (8)

[What about the “mix” of trial subjects?]

Then, of course, I was involved in one of the first male-only trials. The Coronary Drug Project – males only. I’m talking about m-a-l-e. . .You know, it’s sort of like a sin, so to speak. . . I guess I believe, in general, that there’s a hell of a lot more in common between men and women than there is different. So, you know, the notion that when . . you hear about these prevention trials – aspirin, the Physician’s Health Study. It gets knocked around because it involved only males, which is due more to the study section than to the investigators. But, nonetheless, when you hear about that you say, “That’s almost certainly generalizable to women because. . . ” But no, no, they say, we need trials in women.

[By law now we require minorities and women in all studies.]

Right. So I don’t know. So we make much too much of that I think. And I think it takes us to places we ought not to be going. The idea of selective enrollment or focused enrollment or whatever, I think is unethical. The notion, for example, of sort of trying to do a trial in which there’s a quota, take the simple case of males versus females. If I’m doing a treatment trial and I say, “I want a quota of so many males and so many females,” what do I have to do? When I’ve reached my quota I have to say, “No, no, no, we don’t need any more of you. Rather, we want some of you.” I think that poses some pretty serious ethical questions. And as I said, makes you go places you ought not to be going.

[Almost the mirror words of Roy Dawber who was complaining . . . that they forced on him a 2/3 sample in Framingham. He says, “How do you deal with the community? Try lining up the doctors when you say: One out of 3 we’re going to exclude.” It’s the same issue.]

Well again, it’s sort of the politics versus science, I suppose. Are we any smarter now than we were then? (9)

Intent to Treat:

It seems to me, over this period of time from the UGDP to present, analysis philosophy gets pretty much more sharpened. You know, the once-randomized, counted, sort of business. But, if you read the published literature it’s not always followed. The high-quality, big-scale trials certainly pretty much stick to that principle. Once randomized you’re counted. The “intent to treat” analysis. (9)

The Day of More Jobs than Good People:

When I went to college I wanted to build highways. I was going to be a civil engineer. But soon after I got to the University of Minnesota I saw these engineers running around who sort of looked like they were in the fifth dimension carrying their slide rules. I said, “Hell, I don’t think I want to do this.” So I ended up with a liberal arts education and took a biostatistics course in my senior year. You know him, old Vern Weckwerth. Do you know him? Yeah, well, God bless him. I guess he must have been a TA or something for Ellen Trelor. Ellen Trelor was teaching the biostatistics course. So he came to me one day and said, “Pete Bearman wants to see you.” I was very apprehensive. He said, “Oh, no, no, no. It’s all right. Don’t worry about it.”

So I go and talk to Pete Bearman and the end result of that was he wanted me to accept this fellowship. I said, “Let me see if I’ve got this straight. You’re going to pay me to study biostatistics when I don’t really know anything about mathematics? It doesn’t look like a very good deal from your point of view.” And you know Pete. Pete says, “You just leave the worrying to me. I’ll do the worrying. You just do the studying.” The rest is history.

Then when I’m working on my PhD which has nothing to do with clinical trials, Chris Klimt shows up there. So he’s looking for people to work on this study, UGDP, which it ultimately became. He goes to Pete Bearman who gives him the names of four people that he thought he ought to interview. So Chris Klimt interviewed me. Pete told these people to go talk to Chris Klimt, so they went. But I didn’t go. I mean I was interested in finishing my dissertation so I said, “I want to get done with this before I do that.” So I didn’t go.

Well, Chris Klimt, being the sort of guy he is, he had four people on the list and he saw only three. So he goes back to Pete Bearman and says, “I haven’t seen Meinert, I want to see Meinert.” So Pete catches me in the hall and says, “You go talk to Chris Klimt.” I said, “Well, I’m not interested.” He said, “You go talk anyway. It’s rude to not talk.” So I said, “Ok.” So I went and talked and the rest is history. (12)

How did Klimt-Meinert Company get from Minnesota to Maryland?

Chris Klimt, now he’s in the Department of Epidemiology and I’m working with Chris in the Department of Epidemiology and we’re up there on the 11th floor of Mayo. And Chris wants a longer phone cord for his phone. So he goes to administration and says he wants a longer cord. There’s a certain bureaucracy at every university and certainly there’s one at the University of Minnesota. “Well, you have to order it and you have to do this and you have to do that, and so on.” Of course, Chris, not necessarily the most patient guy in the world, decided after making a couple of these attempts to take matters into his own hands. So he went to the equivalent of whatever it was — before the days of Radio Shack, but the equivalent—he goes and finds a phone cord and installs it himself. This ultimately resulted in a tremendous blow-up between Chris Klimt and Len Schuman. Because it was a violation of whatever.

The next thing I know one night Chris is at our house and he says, Curt, I’m going to Maryland. I’m leaving and I’d like for you to come along.” I don’t want to go to Maryland.

Knowing the personalities, it would only be a question of time before there would be some incident where this would happen. But that was the event – the telephone cord – that precipitated it. (13)

What else did we learn from UGDP?

The interesting thing, I suppose, about the UDGP. . .You know the truth is that neither Klimt nor I knew anything about what we were doing. So you’re doing a trial. You’re smart enough to know certain things but you’re sort of making it up as you go. That’s both fun and it’s terrifying at the same time.

You know one of the stories I remember out of the UGDP, because I constructed the randomization schedule. I was smart enough to have written that. I said, “I had better write this stuff down” – how you construct a randomization schedule, the recipe, where you started in the random numbers and, you know, . . So now I write all this stuff down and we proceed. Now 10 years later, and it all hits the fan and the world doesn’t like the results and obviously if the world doesn’t like the results it’s got to be something wrong with you. It can’t be true, as everybody knows this stuff works.

So now you’ve got this Blue Ribbon Biometrics Committee that comes to review. There they come and I don’t remember all of them but Paul Meiers is certainly one of them and various other people. What’s the first thing they want to know when they get there? “Show us your randomization schedule. Show us the recipe.”

Now I’m really quite proud because not only was it written down, but we were able to find it 10 years after it was written. So I rather proudly paraded this thing in front of the committee and gave it to them and waited for them to hand out accolades or whatever. And I look around and they looked a little confused. I asked what the problem is. They said, “We don’t understand the first paragraph.” I thought how dumb can they be? So now I pick up and start reading. It wasn’t in English anymore. Somehow or other between then and now, surely somebody came along and sort of rewrote it, because it now seemed like gibberish. I’ve used that story lots of times when I teach. I say, “The lesson is that it’s not enough to just write it down, you’ve got to give it to somebody else before you put it away and see if they can understand it.” And of course, I wrote it down and I understood it when I wrote it. But nobody else did. (16)

Blind Data Monitoring of Trials:

I’ll tell you another thing, Henry, that I’m not convinced that what we’re doing today is better than what we did in the UGDP with regard to monitoring, or treatment effects monitoring, or whatever you call that. Because what did we do there? Well, we, the investigators looked at the data. There wasn’t any “data monitoring committee” or anything else. It was pretty clear that if you’re going on this long [journey] somebody ought to be looking, so let’s look.

So you look and it was the steering committee that did that and ultimately the story is we stopped tolbutamide because it wasn’t doing any good and so on and so forth. Of course, now, today, you would regard that as: “What?” That investigator saw the results!” I’ll tell you right now that if I had the choice between that and this sort of apartheid kind of treatment-effects monitoring, totally insulated from the investigators—those people sitting on the committee really don’t know the protocol— I’ll take the stuff done in the UGDP. Because I think it’s better.

There is such dogma today with regard to these sorts of things. They either think I’m prehistoric or crazy or both. You’re going to have to show me that I’m wrong. . . I keep saying that it’s more important to have competency in the monitoring committee than objectivity. But all the emphasis now goes the other way. Everybody wants objectivity and . . . this is where this masking comes in, and the stopping rule business, and, you know, the number of looks at the data.

I mean, I got to tell you, and certainly you were involved with them, you can go through your life and you can mention a few people that have had impact on you in various ways. Certainly Jerry Cornfield is one of those guys. Because he says, “How can you spoil data by looking at it?” And of course it’s true. How the hell can you spoil data by looking at it? Yet that’s what the concern is. “Well, you’ve got to specify the number of looks you’re going to make before you start, no more, no less, because it will screw up the p-value.”

What the hell are we talking about here? I mean, who are we protecting when we do this? Where the emphasis goes to the protection somehow of the p-values and not the patients.

[The CDP was a pretty good compromise on those issues, wasn’t it? You had members of the investigators on data monitoring.]

Exactly. You talk, by the way, about other people, but certainly Jerry Stamler is somebody who taught me, his organization, the way he would run a meeting. (17)

More Trial Management:

But, you remember in that [Coronary Drug Project] structure that was sort of a joint committee, you were on it, Jerry Stamler was on it, I was on it, several other people from within the study and some outsiders . . reviewed the data and then sent recommendations to what was called the Policy Board. And the Policy Board either accepted those recommendations or rejected them. And if you remember with regard to the high-dose estrogen treatment group, the Monitoring Committee said, “Ok, we ought to stop it in these high-risk men.” And when we sent that to the Policy Board the Policy Board said, “We don’t see any good evidence that says this effect is there and not here. You ought to stop it in everybody.” And I think that was a good decision. But there was some advantage, it seemed to me, to that division of labor but that structure has largely gone away.

How does it normally work [these days]? Well, it depends a little bit on how the study is funded. Let’s say it’s a NHLBI-funded study, contract funded. Well, I can guarantee you that that committee is going to be appointed by the Director of the NHLBI, that’s going to report to the Director of the NHBLI, that the routine from there on in of how these things get back to the investigator is going to be unclear. But that body is beholden to the Institutes, going to report to the Institute, they’re going to be appointed, without advice and consent from the investigators, and, in fact, speaking of your walk away from MRFIT. I walked away from a trial we’re doing here because I got in an argument with NHBL I kept saying, “Ok, it’s Ok for you to do that but then I want an iron-clad guarantee that says those recommendations get passed without fail through the investigators. I don’t want a structure in which they could be pigeon-holed or whatever.” Well, Lenfant says, “We would always do that.” I said, “Well, fine. Put it in writing and I’ll stay.” “We always do that,” [he said]. I said, “Put it in writing.” And it never happened, so I walked.

[Now they want to design it, they want to run it, and they want to publish it. It’s really undermined the basic infrastructure of the science and destroyed multi-center RO1s, as you predicted years ago.] (23)

Speak of Meta-analysis.

Look, it seems to me, I mean, I believe in analysis, therefore I have to believe in meta-analysis because it is analysis. But I don’t believe I was put here on earth to simply feed the meta-analysts!

[Peto and Co. and Cochrane Company]

Right. My objection, I suppose, to meta-analysis doesn’t have to do with the notion of meta-analysis. It has to do with at times almost a certain amount of almost arrogance with regard to what your duty as a trialist is. Well, they think your duty as a trialist is to simply provide decent data for their meta-analysts. . . . I’d rather be a meta-analyst than a trialist. It’s a lot easier. It’s a cleaner industry.

I don’t know if you noticed, but I published a paper in which I invented a little game called “muta-analysis.” Well, muta-analysis is the redesign of old studies. So I did a muta-analysis and it had to do with this gender stuff in which I redesigned MRFIT to involve women, and redesigned the Physician’s Health Study to involve women. And used the cost figures for those studies to see where this would take you, for example. You know, if you play the MRFIT game in regard to women and if you take literally what Congress mandated, meaning that you’ve got to come up with equal precision of results for women and men, and for blacks and whites, and so on, it drives the costs of MRFIT to over a billion dollars. It’s in Statistics and Medicine.

Anyway, I’ve said what I believe there. But one thing I believe in firmly but I’m in a minority, I think we ought to have planned meta-analyses. In other words, designed studies amenable to later meta analysis. It seems to me that when you’re monitoring, there ought to be certain classes of studies in which you are by definition pooling results. . . For small trials we ought to be more or less insisting on some form of “meta-analysis.” Because we behave as if each trial is independent onto itself and no other information. The Hormone Replacement Trial is part of a much larger business. (27)

Minority Quotas:

MC:. . the NIH sort of waltzed around the Congressional mandate for inclusion of minorities and women. If you read the mandate, what Congress said basically is that you have to design this thing so that you can tell with equal power, so to speak, equal precision in minorities. The NIH ultimately started talking instead about being able to do a ‘valid analysis.’ That was their euphemism for sort of saying, to soften profoundly the . . literal requirements. Because if you took the requirements literally you’d end up having to have thousands and thousands of people.

HB: I was on the Advisory Council at the time that ruling first came out and I argued that we should design studies to take special care of the specific scientific questions in minorities, not inflict quotas on all studies.

MC: Yes. But, of course, you were in a minority. That’s the trouble it seems to me. (29)

More Trial Controls; a Dialog:

HB: Well now they’ve got the rule that you can’t increase your budget more than 10% a year, that really helps a clinical trial or epidemiological study! The start-up year is here and your next year needs twice the funding.

MC: Well not only that, but then they’ve also got this law that if you want to submit an application over $500,000 you’ve got to have permission even to submit it.

HB: And that’s very arbitrary. They’ve got all sorts of devices of control and discrimination.

So, are you going to be an angry old man and do something about it or just cash out at this stage? How do you turn this around? Instead of [NIH]administering for the investigators, they’re running the show and they’re arrogant about it often.

MC: The way I entertain myself is to try and get the investigators of the world to unite, so to speak. And I don’t know quite what that means. It seems to me if we want this to work and we want this to be better and not worse, then we have to individually and collectively, have to put energy into it and sort of stand up and be counted. And that’s not happening for different reasons. Not the least of which is who needs it, I mean I’m busy.

And unless you’re sort of old and crusty you say, “This is not necessarily going to do any good for my career.” As the creator of the Society for Clinical Trials and that sort of thing, I’m profoundly disappointed in the Society for Clinical Trials because it’s an inept organization. It’s just passive. You’ve got to have a certain amount of guts. You know I write a lot of letters and there are times when people see you coming and they say, “Oh no, not you again.” You know it’s not said in a bad way, but you sort of wonder. . . I think that you have to wait for something bad to happen before you can have anything good happen. I’m just sort of waiting for some sort of celebrated event someplace where some trial goes on too long or does something stupid because the . . . data monitoring committee is so separated from everybody else that it’s incompetent.

HB: Like the CIA/FBI and terrorism?

MC: Right. Then something will happen. But until then, that’s the model, that’s a good model and we don’t need to do anything about it. . . I don’t know if it will happen on my watch, but sooner or later there will be people, ethicists who will say, “Can you imagine when we were doing trials when people enrolled in the trials didn’t know what the results were?” Because sooner or later results are going to be in real time.

Then . . they say, “Well that will be the end of the trials.” That isn’t going to be the end of trials. It will just change them. And in some sense it’s for the better. You [we] were close to that in the area of AIDS where people would come and steal your data monitoring report and put it on the internet so that sick people knew what the results were.

Think about it for a while. Intellectually, what’s wrong with that? Pigs is pigs and data is data so what the hell? This is what it is now. I think that will happen. I don’t know when. (38)

Feinstein’s direct attack on UGDP, as Consultant for the Drug Industry:

HB: A sore subject but just a word about Al Feinstein. He has died recently and there’s an award in his name by the American College of Physicians. All the Feinstein business floated back to my mind last week when I wrote a letter to nominate Jack Farquhar [for a Feinstein Award]. Now they’re eulogizing Feinstein and he was a bright SOB.

MC: No doubt about it.

HB: But all I remember is that he accused Minnesota of misreading electrocardiograms or rereading electrocardiograms and I interpreted that as accusing me of being dishonest.

MC: Right. In other words, he claimed basically you didn’t get the right results the first time so then you revised the code. You went back and reread them so that you could end of up with what you wanted.

HB: That was his accusation.

MC: Which, of course, was only one of dozens of accusations made. . . This was Feinstein’s way of operating.

HB: Just a tactic – attack!

MC: Right. You discover after a while it wasn’t necessarily personal, but it was just his tactic. (41)

The Supreme Court Considers Ownership of UGDP Data

MC: You mean ownership in regard to federally sponsored studies?

HB: People were suing to get access to the UGDP or CDP data.

MC: Yeah. I got this game I always threaten to produce – I never will, of course – but it’s called “Trivial Pursuits for Clinical Trials.”

HB: Wonderful idea.

MC: And one of the questions in there is, “What do Henry Kissinger and the UGDP have in common?”

Well, the answer to the question was that the case for access to information under the Freedom of Information Act, that those two cases are joined. People wanted Henry Kissinger’s telephone logs and other people wanted the “raw data” of the UGDP. And those two cases sort of went through courts and they coalesced in front of the Supreme Court. The Supreme Court heard both those cases as one case, so to speak. So the Supreme Court ruled simultaneously on Henry Kissinger’s telephone and the UGDP. (But, that was largely because . . the fact was that even before that case came we had published in Diabetes the data listing for every piece of the stuff in our dataset except for names and a few other things. So if you looked in the back of one of those supplements it’s all in there. ) What was wanted, so to speak, was the entire “medical record”.

HB: To see if there was negligence, etc?

MC: Right. Because obviously, there’s got to be something there that you didn’t record because we know that this stuff [Tolbutamide] works.

That immediately brings the government and the university in on the side of the UGDP and Klimt and Company because medical records are medical records, they’re not public documents. But the interesting thing, I think, about that case was that if you read the court ruling it’s 1) a narrow ruling. I mean you can’t generalize it to anything else. And 2) it was a 5-4 vote and five of the Justices said, “No, the intent of Congress is NOT to write a Freedom of Information Act that would obligate Kissinger to give up his logs or obligate investigators into giving up their raw data.” They just sidestepped the issue. It was a narrow ruling.

HB: Interesting. Has it been tested again?

MC: I don’t know. I should probably have posted that frigging label insert with the black box on the wall some place because it’s 14 years from the time that the UGDP results come up before the black box appears on the label insert. Fourteen years! You know the black box means ‘Be careful, maybe there are cardiovascular risks’. It took exactly 14 years for that to happen.

HB: For Tolbutamide?

MC: Yeah. I always tell students, “If you’re practicing medicine by the drug label insert, be careful. Because it runs behind times.”

HB: Fourteen years of bureaucratic foot-dragging.

Trial Management:

You know, I told Pete Bearman – I regard Pete as my professional father. I said, “You know, Pete, my degree in psychology as an undergraduate is more beneficial to me than all my stuff in biostatistics.” Now that’s a slight overstatement, but it turns out that you spend a lot of time in the business of clinical trials on organization and group dynamics. I tell people, at a coordinating center you may come to the first meeting and it may involve 50 clinics and you might know five people there. Because it’s a different area. I say, “Size up where people sit, look at their body language. You can learn a lot about who’s doing what to whom by seeing who they talk to …”(56)