The argument for re-evaluating the evidence that informs clinical decision making

[background music]

Dr. Jamie Meuser:  You're listening to "Clinically Speaking." A podcast that explores the past, present and future of evidence based medicine in primary care. Brought to you by the Center for Effective Practice. I'm Jamie Meuser, a family physician practicing home palliative care in Toronto.

Christine Papoushek:  I'm Christine Papoushek, a pharmacist practicing at a family health team in Toronto.

If we have doubts about the basic quality and credibility of the scientific studies that are the core elements of clinical practice guidelines, then how can we rely on them to support our clinical decision making?

Dr. Meuser:  In our last episode, we heard a strong critique of the quality of some of the core evidence used in psychiatry. Not surprisingly, doubts about our ability to rely on published studies, let alone clinical practice guidelines are certainly not confined to any particular branch of medicine.

Christine:  Today's guest, Dr. Nav Persaud, is a respected clinician scientist who has, since residency, consistently and cogently questioned the value of the evidence underpinning many common, accepted clinical practices.

Dr. Meuser:  His criticism takes us down some unusual directions that may surprise you. We also think they also provide some ideas about possible future directions for EBM.

We are sitting down today with Dr. Nav Persaud. Nav is a family doctor at the St. Michael's Family Health Team in Toronto and a scientist and researcher at the Li Ka Shing Research Centre associated with St. Michael's Hospital in Toronto.

Nav, thank you and welcome. As we've discussed, this podcast series that we're doing is broadly scoped around the use of the place of the future of evidence in clinical decision making in medicine.

We'll get into, in a little bit, your particular interest in, and role in that enterprise. I would like to start, though, by asking you your take on the current state of evidence based medicine.

Dr. Nav Persaud:  Most research done today is of low value. The purpose of health research is ultimately to promote the health of people. When you look at most of the individual research studies being done today, they would have little or no benefit to people.

I'm thinking largely of trials where one medication is trialed against another, one antidepressant is tested against another. Most of that time those trials find little or no important differences between two treatments, so why are all of these studies being done?

There was recently a meta analysis of head to head trials of antidepressants. It included more than 500 trials where antidepressants had been trialed against each other. The main conclusion was that there didn't seem to be a really big difference between them.

In fact, what I thought was the most striking finding, is that a medication appeared to be better when it was new. The very same medication appeared to more effective and have fewer side effects when it was introduced as the new product and it was trialed against other ones that were older, but then when that very same medicine became an old product, it looked worse.

That shows there is a huge problem with the validity of studies being done. The answer is definitely not, in my view, to do another 500 head to head trials of antidepressants, but if you look at what's being done, it probably won't actually be that long before there are another 500.

Dr. Meuser:  If I take your point correctly, a major threat, then, to the usefulness of evidence in the early 21st century to clinical decision making in medicine is simply that what goes into the crafting of it is suspect sometimes. It's not done with the intent of helping clinical decision making.

Dr. Persaud:  Exactly. Those sorts of trials seem to be done mostly because a company wants to sell a different antidepressant. It may not be any worse than the other antidepressants. It may not be problematic, but it also isn't advancing the care of people with anxiety or depression.

Dr. Meuser:  In your view, what would constitute high quality evidence?

Dr. Persaud:  Testing interventions, could actually have an important difference from existing interventions that we have. Testing medications that really are new, not just me too drugs that are in the same class, do the same thing as existing treatments. Where there's at least a chance that there would be a real benefit, I would consider that to be higher value research.

Other things, like looking at how medications work in the context of other medications. It is one thing to demonstrate that an antidepressant is effective, another thing to demonstrate that it's effective in the people who are typically using it, people with other chronic diseases.

Looking for those sorts of interactions in real world effects of medicines, I think, is more useful than these efficacy studies of medications that are very similar to each other.

Sometime they are, if fact, the same molecule being trialed against itself. These enantiomers of one proton pump inhibitor versus another enantiomer where it's almost impossible to imagine how there could be any important difference between them.

Dr. Meuser:  You're saying save the effort for things that are truly different from each other, or that they have some innate difference from each other that...?

Dr. Persaud:  I'm sure that people are arguing that there's some reason that there's an enantiomer of [inaudible 5:05] is going to be better than that enantiomer, but if there really was a big difference we would have to rewrite the laws of physics and chemistry.

It's unlikely we're going to discover something like that in a clinical trial. Given that we have limited resources, given that there are always going to be risks and downsides to the patients participating in these studies, it probably makes more sense to only do studies when there could be a value.

Actually if you go back to the fundamentals of the ethical principles for doing clinical research, there has to be some prospect that care is going to be improved before a study could be ethical regardless of how else it is conducted. I would argue that many of the clinical trials being done today are not ethical.

Dr. Meuser:  You've [inaudible 6:38] a position. That's a pretty strong one. Can you describe some of the projects you're working on these days, that if realized will allow clinicians to have a higher level of confidence than the evidence that is put in front of them for clinical decision making?

Dr. Persaud:  One focus now is on essential medicine's list. The idea is we should be able to write down on one piece of paper the most important medicines that a family doctor should know about and should have available to them. Also, that patients should have access to. That may sound to some people as a pretty lofty goal.

Consider the alternative, what if after decades of doing clinical research, after tens of thousands of clinical trials we couldn't do that. Someone would say, "Well, what are you doing then in healthcare if you haven't been able to figure out which are the most important treatments? What is all that research you're doing for?"

I see creating an Essential Medicines List first as a way to improve clinical practice because the reality is healthcare providers, physicians, family physicians, in particular, cannot possibly keep track of thousands of medicines, can't know the indications for them, the adverse effects all of their interaction.

It's just impossible. We know that doctors aren't going to take 15 minutes to look that up each time they want to prescribe a medication. It is possible that a individual clinician can learn about a hundred or two hundred medicines that they commonly use. Most already do that, but they may or may not be the best ones to prescribe.

We know that doctors can be influenced, not just by the evidence but by other actors who are eager to influence doctors. That is really the way to focus clinical practice. It also sets us up to do high value research. We have the current state, the medications, and interventions that are currently being used.

Then if a medication is to be added, if there's a question about whether it should be added obviously that needs to be based on research. If there really is an antidepressant that's better than the ones that are on our Essential Medicines List, then it should be added, but we'd have to do research to find that out. We don't necessarily need to do research on a dozen different antidepressants.

Dr. Meuser:  Can you tie the idea of having a finite list of essential medicines back to the point you made earlier about the evidence just not being that good for many of the choices that we currently have?

Dr. Persaud:  First of all, I think one main issue is there are just too many studies of too many medicines going on. Having an Essential Medicines List where we had a handful of antidepressants on that list, two or three. Then we would focus the question.

The question is if someone is saying there's this new antidepressant that we should be prescribing. The question is obviously, "Is it better than the ones that we currently have?" You limit the number of studies that you need to do. Then hopefully you can pool resources to do those studies properly, and importantly you can try to replicate findings.

If a pharmaceutical company does a clinical trial and finds that a medication is more effective than another, before we added to the list because there aren't that many decisions we have to make, someone else can try and replicate that finding. If it is replicated then absolutely the medication should be added.

That's not what we're doing now. Now what happens is a study is done, medication gets approved, it gets marketed, there's a promotion because it's new and expensive, so there's an incentive to get doctors to prescribe it. Then there's all of this excitement. I'm just thinking about the prescribing of antipsychotics that I see.

I can hardly keep up with the new antipsychotics being prescribed. Then when I sit down and try to understand first of all what they are, because usually they're being referred to by their brand names. You don't even know what the medication is.

When you find out what the medication is actually called and you look at the evidence, it doesn't seem to be any better than the older antipsychotics. That shows how quickly this low value research gets translated into changes in clinical practice, that don't    as far as I can see benefit patients.

Dr. Meuser:  You've been tied to some efforts in the re review of past evidence has been used to include particular drugs. In many cases are particular interventions as the standard of care.

Do you want to talk a little bit about where that effort came from? I said it's very parallel to what you've already discussed but some of the things you found out along the way?

Dr. Persaud:  Yeah. It came from a mistake. I used to prescribe Diclectin, Doxylamine, and Pyridoxine for nausea and vomiting during pregnancy. I prescribed it routinely, just like many other clinicians in Canada have prescribed it and still prescribe it today.

Thankfully, I had a patient come in and question the prescription. She said, "Are you sure that I should take this?" I said, "Yeah, of course, I'm sure. It's recommended as the first line treatment. I prescribe it all of the time."

She had some misgivings and after she left, I went back to the guidelines and I looked at what it was based on. I wasn't sure that everything I said was in fact true.

The guideline was ridiculous. Instead of referring back to a systematic review or even a clinical trial, it referred back to the product monograph. A document made by the drug company that sells the medicine. Not the basis for a first line recommendation.

Over the years, I have been trying to understand why this medication is so strongly recommended. The answer is not to be found in the published literature. When I started this, there were no studies in the published literature that would demonstrate that this medication was effective. Not even that it was better than a placebo.

Certainly not that it was better than other treatments for nausea and vomiting that can be used during pregnancy.

Over time I have tried to obtain both from the company and from Health Canada all of the information about the clinical effects of this medication. I was shocked to find out that that sort of information is considered confidential    was not disclosed by the company that makes it. It's called Duchesnay, based in Quebec. It was also initially not disclosed by Health Canada.

In response to my freedom of information request, they sent me a whole bunch of redacted pages and marked them as confidential. This is not    to be clear detailed information about where this product is manufactured, this was information about the effectiveness and safety of the medication.

Over time, after a change in the law    this is now more than five years down the line    eventually we got access to this detailed clinical study report. An hour interpretation of the findings is that they're consistent with the medication not being effective at all.

Dr. Meuser:  Have you been able to come to any conclusions along the way then based on this? This has certainly been influential work you've done, not just for that product and that clinical condition but has triggered a lot of similar efforts along the way.

It raises for me the question of how good evidence has to be to be useful, to be applicable for people like us whose job it is to make decisions every day?

Dr. Persaud:  First of all, I don't know that it has been that influential. It seems like doctors are pretty happy to keep prescribing this medication despite the information that's been uncovered recently.

That is partially an answer to your question. In actuality, prescribing is not necessarily based on the quality of this trial. It's based on a lot of other factors. One of them that I think is at play to a large extent here is habit and familiarity.

Doctors and other healthcare providers like midwives and nurse practicians have been prescribing this medication for a long time. They know the starting dose. They know the maximum dose and the instructions you're supposed to give. They haven't encountered problems. They haven't had many women come back with concerns.

I have had senior doctors say to me, "You're telling me that this medication I've been prescribing for the last 20 years doesn't work. I've been prescribing since before you were practicing medicine. Before you went to medical school. Maybe you weren't born, I've been prescribing this medication and now you're telling me..."

Dr. Meuser:  Sunny boy.

[laughter]

Dr. Persaud:  They don't have detailed questions about whether there was allocation concealment in the trial. That's really not the conversation that we're having.

There's a pretty big disconnect between details of the quality of clinical trials and what medications people are prescribed by their doctor.

I don't actually think that the main issue is that we need to improve the technical quality of clinical trials. Obviously, we need to have trials that are technically sound and better designed properly, but that's not the major problem with evidence based medicine today.

Dr. Meuser:  Your word has brought into question    for some people in fact whether a particular drug which was seen as a standard of practice actually should be and have them potentially reconsider a decision about what they are going to recommend the patients they see.

I wonder if this will bring into question, for them or others, the whole notion of clinical practice guidelines for being away of packaging information. Whether there are consequences to lowering confidence in clinical practice guidelines in general.

Do you think we should have less confidence than we do in guidelines?

Dr. Persaud:  I don't know that the confidence is high right now but it probably is too high. I think that many of the clinical practice guidelines that exist today don't address the major issues, the major uncertainties that doctors have and that would have implications for care.

I could give a few examples. Let's start with the one we've already been talking about, the nausea and vomiting during pregnancy.

If there were no clinical practice guidelines in Canada, what would've happened is we likely would have treated nausea and vomiting during pregnancy the way we would treat allergies during pregnancy. We would take treatments that are effective for allergies outside of pregnancy and we would use them during pregnancy, provided that the safety of those medications during pregnancy had been assessed.

We could inform women about the risks during pregnancy. In fact, we tend to prescribe the same medications for women who have allergy symptoms during pregnancy as outside of pregnancy.

Those medications overlap with the medications we could prescribe for nausea and vomiting. If you look at what is prescribed in other countries, that's what they do. They prescribe medications like Metoclopramide that work for nausea and vomiting outside of pregnancy during pregnancy.

It's not a big surprise that that medication could be effective during pregnancy because pregnant women are still human beings, when they have nausea and vomiting, medications that work for other human beings help too.

We could've had a more rational approach but what has happened and is happening today is people say, "Well, this is the nausea and vomiting during pregnancy pill. When a woman comes in with nausea and vomiting during pregnancy, this is what I have to prescribe."

There's that herd mentality, "I feel safe because all my colleagues prescribed it. If someone were to criticize me...if a woman did have a child with a malformation and she came back and tried to say it was this medicine, I would just say I was prescribing what everyone else prescribed. That would be safe."

Care would actually be more rational and it would've been better in the case of nausea and vomiting during pregnancy if we just had zero clinical practice guidelines.

There is an open question about the relationship between the manufacturer of this medication, Duchesnay, and the two organizations that both issued clinical practice guidelines with this medication at the front line.

Those are the Society of Obstetricians and Gynecologists and the Motherisk Program at The Hospital for Sick Children, both of which over the years have received funding from Duchesnay.

Chronic non cancer pain is another example. If you look at the earlier guidelines, they recommended maximum or watchful doses of 200 milligrams of morphine or equivalent per day. In retrospect, that was based on very little.

Looks like nothing really and it was seen as an advance because the prior position that had been disseminated in various ways throughout the profession was that opioids had no ceiling dose.

Even in the case of non cancer pain, non palliative pain, you could go as high as you like. That's a pretty astounding view, actually, for any medication    the suggestion that the dose didn't matter, you could go as high as you like. Especially for a medication that's fatal in overdose, seems like dangerous and hazardous prescribing.

That was an example where a guideline recommendation, it was very important and had important implications for people's health. In fact, important implications for the death rate, related to opioids in Canada, and maybe even for the life expectancy for Canadians we're seeing now. That sort of recommendation was based on very little.

Now, there are current attempts to try and walk back from that but it's also very difficult just based on the way that clinicians operate. The particulars of this medication, where once a patient is on a high dose, it becomes very difficult to lower the dose.

You look at other examples, like the guidelines for diabetes. They tend to list every medication that's out there and make vague statements about how to select them. Some of the national guidelines for diabetes care just list the medications in alphabetical order because they couldn't think of any way to prioritize them.

Even though insulin has a different history and standing than these newer treatments do, they're listed in alphabetical order.

In many ways the actual decisions that a clinician has to make when a patient is in the room aren't helpfully informed by recommendations from clinical practice guidelines. They're not critical and reliable sources of information.

Dr. Meuser:  This takes us back to the point you started with around the Essential Medicines List. Which is    as I understand it is if we can't count on clinical practice guidelines and the packaging of evidence the CPGs would represent, if in fact what they offer to clinicians is scant help in making important decisions about what they're going to do next with their patient.

Then maybe what we should do is stick with medicines for which there is solid evidence and let everything else that comes along compare itself to the solidness of solid evidence.

Dr. Persaud:  Exactly. What gets lost in these clinical practice guidelines is we do have very effective medications. We have life changing medications for HIV Aids, for infectious diseases, for high blood pressure, for diabetes. That gets lost when there are these long list of medications.

The other thing that gets lost is that there are conditions where there aren't effective treatments. There are clinical practice guidelines for the treatment of overactive bladder. They will also mention a number of medications that you can use and they'll compare the side effects of them and the effectiveness.

When you look at the evidence, it doesn't look like any of these medications have a clinically important benefit. A benefit that a patient would actually notice and would want to accept the risks that are present with any medication.

Dr. Meuser:  It's pretend guidance.

Dr. Persaud:  Yeah. It would really be a sham guideline there where there in fact is not an effective medical treatment.

Dr. Meuser:  The Essential Medicine List feels to me to be something not many people are talking about yet in this world. At least not in the context of evidence based clinical decision making.

Dr. Persaud:  It's really striking to me that there is a disconnect between essential medicines and evidence based medicine. Those two worlds really haven't spoken to each other at all.

That's something I've been thinking through and wondering if there's a contribution to be made there because Essential Medicine List would be a way to apply all of the great methodological work that has been done in evidence based medicine.

On the other hand, looking at it the other way, I think evidence based medicine is mostly spinning its wheels today. The details of how to conduct a clinical trial today are being applied to low value or wasteful studies. Testing two similar antidepressants against each other. That is a useless thing to do.

On top of that, doing those trials and people are synthesizing all of that useless information and creating guidelines based on useless information. This big and really efficient and well thought out machine of evidence based medicine is being miss applied to answering useless questions. It could be put to much better use to answer the important question of which are the medicines that people should be using.

Dr. Meuser:  Besides the Essential Medicine List, any other opportunities that have emerged from the things that you've been looking at and thinking about for improving access to and use of evidence for clinical practice?

Dr. Persaud:  One is going to the source and determining whether the trials being done today are ethical. That is actually what has worked the literature and worked...

Dr. Meuser:  What do you mean by ethical?

Dr. Persaud:  I mean, "Are they answering an important question or not?"

This mountain of clinical trials being done answering unimportant questions is created. People feel they need to synthesize it and they need to make recommendations because nobody can sort through which of them is better. We could stop synthesizing that and we could stop the clinical practice guidelines. That would be a useful thing to do.

Even better, just to stop those trials being done. Let's take those fundamental ethical principles seriously and ask ourselves whether or not these clinical trials are ethical.

Dr. Meuser:  A couple of points that have come up in talking to others about evidence based medicine and clinical practice guidelines. Any thoughts about the role of patients? Your Diclectin example started with a patient question.

I wonder if there is anything that you've been thinking about or you have come across that speaks to an important role for patients in either creation of new evidence that we can use and apply to other people like them or an assessment of the compilation of evidence. Should patients be involved for instance, at any point in the clinical practice guidelines process?

Dr. Persaud:  I completely agree that it's patients and members of the public who are going to save us here. The first place it should be involved is in prioritization. Determining what are the most important research questions we should be asking and answering.

In my research, we are working to also involve members of the public and patients in designing studies so that we are measuring the right outcomes, measuring outcomes that are important to people.

Not just ones that can convince a clinician to prescribe a medication, but that would make a patient actually want to take the medication. I also think that there is an important role for patients in synthesizing evidence, in creating clinical practice guidelines.

It's both in terms of crafting the recommendations properly but again making sure that the recommendations are on issues that are important to patients.

[background music]

Christine:  Thank you for listening to this episode of Clinically Speaking. It was produced and edited by Pippi Scott Meuser at the Center for Effective Practice.

Dr. Meuser:  Special thanks to our guest this episode, Nav Persaud, for taking time out of his busy schedule to talk with us. We'd love to hear from you. If there's a specific topic you'd like us to cover, please feel free to send us an email at info@CEP.heath or come and find us on Twitter @CEPHealth.

Christine:  Don't forget to subscribe on iTunes, our website, or wherever else you get your podcasts.