Who influences the evidence in evidence-based medicine?

 [background music]

Dr. Jamie Meuser:  You're listening to "Clinically Speaking." A podcast that explores the past, present and future of evidence Based Medicine in primary care. Brought to you by the Center for Effective Practice. I'm Jamie Meuser, a family physician practicing home palliative care in Toronto.

Christine Papoushek:  I'm Christine Papoushek, a pharmacist practicing at a family health team in Toronto.

We rely on clinical practice guidelines and systematic reviews to package and deliver us the evidence we need to help us make the most informed and best decisions for our patients. What happens when that so called evidence we rely on isn't reliable at all?

Dr. Meuser:  Psychiatry and mental healthcare in general have long suffered from nebulous, malleable and ever changing evidence. This is also the place where some of the earliest and most seminal work has been done to re examine and, where necessary, to correct tainted evidence that has long informed diagnosis and treatment of patients.

Christine:  Today we speak with Dr. Elia Abi Jaude, a psychiatrist at SickKids in Toronto with an academic reputation for critiquing how evidence is used, and sometimes misused, in psychiatry.

He is one in a group of clinicians and researchers from around the world who have taken on the task of re reviewing the accepted evidence that has gone into the assumptions made about how diagnosis and treatment happens in psychiatry.

Dr. Meuser:  He has plenty to say and he certainly doesn't hold back, so stick around.

We're talking today with Elia Abi Jaude. Elia is a psychiatrist at the Hospital for Sick Children in Toronto. Practice, Elia, is primarily child and adolescent these days?

Dr. Elia Abi Jaoude:  These days, yes. I see across the lifespan.

Dr. Meuser:  The other side of your practice is primarily focused on individuals with tics?

Dr. Abi Jaoude:  Yeah, tic disorders and related disorder.

Dr. Meuser:  Then a particular academic interest, as I understand it, in looking deeply at how the degree to which we can rely on the evidence that's used for clinical decision making in psychiatry. What's your reputation amongst your colleagues?

Dr. Abi Jaoude:  Depends who you ask.

[laughter]

Dr. Meuser:  Are you seen as a cynic, a skeptic, a rabble rouser?

Dr. Abi Jaoude:  I would say all of those, I think by people who don't like the establishment being shaken up too much, may be seen as a threat, too much of a radical. The other extreme is people who see me as an advocate and someone who is a progressive and wanting change.

This is an area that I've gotten into in spite of me. This is not an area that I ever planned to get into. In my training, even starting in medical school, especially so in residency, I was asking a lot of fundamental questions that I expected some reassuring answers, but the answers were not addressing some of the very fundamental important questions.

I continued to question things, and as you know, the more you question, the less you can rely on blind faith in terms of what you're learning and what you're being taught.

I gradually got into this field, initially, after my residency training, I was asked to give the odd teaching session and the odd presentation. Then it just grew. I've started even doing academic research along these lines.

Most notably in the last couple years, I was involved in the reanalysis of Study 329. For those who are not familiar with it, this was a drug company funded study, looking at the antidepressant paroxetine in adolescents for depression.

The company concluded that the drug was safe and effective, but there were questions about the study. I and a group of international researchers, we were able to access the raw data, reanalyze the study entirely, we come to the exact opposite conclusion of drug being ineffective and unsafe. Then, as you might imagine, got a lot of attention.

It's something that I've gotten sucked into. It's just you cannot be a practicing clinician, and turn a blind eye to this. It's as important as Evidence Based Medicine is, basically. If we accept that we should make decisions based on evidence, then it behooves us to make sure that the evidence on which we are basing our decisions is sound and independent.

Christine:  Is there anything different about the context of decision making for clinicians dealing with mental health that has influenced EBM practice in this clinical area?

Dr. Abi Jaoude:  I would say yes and no. I always want to point out very quickly this is not a psychiatry issue. Psychiatry has a reputation of, we have our nebulous, diagnostic boundaries, that intersects with culture so much, even law and philosophy. It's complex to make a diagnosis and say, "OK, what is an illness and what's not?"

This applies across medicine. Just because there's a number on a blood pressure reading or a blood sugar level or bone densitometry scan or lipid level doesn't all of a sudden make it objective. There are all sort of complexities that go into determining where the thresholds are.

This is not unique to psychiatry, I cannot over emphasize that. However, I think I see psychiatry as the canary in the mine when it comes to these issues. We have to acknowledge it has been particularly susceptible to these issues. It's not new, it's not recent.

The concerns that are often raised, and again this is not specific to psychiatry, but particular to psychiatry is the medicalization of life. Pathologizing of the human experience.

I need to rush to say often colleagues become uncomfortable when they hear me talking along these lines. They start to bring up the issue of antipsychiatry and whatnot. I'm a practicing psychiatrist. I love my field. There's a lot to be done to help people who come in with psychiatric challenges.

I don't think we should be afraid of the antipsychiatry movement. It's because we don't self critique enough. We've allowed the antipsychiatry movement to do the critiquing themselves. In our field, there's a lot of diagnoses that I think are problematic in terms of the thresholds and how we determine them to them.

Maybe I'll touch on one that's probably going to surprise people and it's very hard to question. Andre Picard did a very good job questioning it in a recent article. That was at "The National Post" or "The Globe" [inaudible 6:19]?

Dr. Meuser:  "Globe and Mail."

Dr. Abi Jaoude:  Globe and Mail. Anyhow, in particular what I'm talking about is depression.

What we call depression today is very different than what was depression a half a century ago. Half a century ago you would see the person you could tell there's something wrong. It was no question about it. This was a severe serious mental illness.

With every iteration of the DSM that threshold has gone lower and lower for what determine as depression. First of all, there was a conflation between what's referred to sometimes as the melancholic depression and people who are struggling because of what's going on in their lives.

Today the numbers that are given...I think we're conflating depression with suffering, but when we follow the current DSM 5 criteria which have become ridiculous, to be honest. Especially now that they've removed the grief exclusion criteria.

Meaning if you meet these criteria for two weeks because you lost your loved one, you can still become someone with a mental illness. Unfortunately, this triggers a very formulaic algorithmic approach to helping the person in front of you. That's overly narrow. It doesn't address what's really going on in their lives.

It floods the system with people who are suffering, but I don't know that these people need to access care in the tertiary care centers and such, and then makes access more difficult to people who have very serious mental illnesses. This is an example of where psychiatry had become particularly problematic.

Dr. Meuser:  It sounds like you're not overly impressed by the use of evidence and formulation of the DSM.

Dr. Abi Jaoude:  No, I think there was the Wild West before Evidence Based Medicine. The Wild West was a combination of culture, tradition, confounding biases that were blind to overreliance on observation the expert opinion and such. I don't know that Evidence Based Medicine has been able to address these things fully.

On top of that in the last three decades or so, the power of the biopharmaceutical industry in generating this so called evidence and in disseminating this so called evidence. The evidence is not just for treatment but even for what we call it a condition.

Let's look at the last hundred years. If we focus on psychiatry what it's been defined by? It's been defined by insulin coma therapy, it's been defined by lobotomies, it's been defined by the psychoanalysis. It was religion. Each of these things becomes, you don't dare to question.

Young children who have been unwell for a few weeks were getting lobotomies. That's how it was. Again it was very hard to question these things. Comes to psychoanalysis we had concepts like refrigerator mothers. It's schizophrenogenic mothers. Think of the damage we've caused with that.

Then move in the last few decades to the rise of the biomedical industry, and it's like the volume housewife [inaudible 9:07]. Everyone's getting volume. Then the rise of the serotonin theory that was based more on pseudoscience than science. What it was is the rise of Prozac. Then move another decade or so and it's the rise of the atypical antipsychotics.

If you look at the evidence for what really defines an atypical from a typical really there's nothing that stands firmly, that neatly separates the two other than that they were on patent when they first came on. All of a sudden there's this volume housewife was went on Prozac, next thing she's on Zyprexa.

If you follow pharmaceutical marketing when you look at the internal industry documents and where they wanted things to go. Then you'll see how the field is actually moving along, it's fascinating. That's where the evidence is today. It's very hard to trust any published papers. The degree of bias, the extent is huge.

I can give you some specific examples just to give you an idea of what I'm talking about. Eric Turner, a psychiatrist in the US, used the Freedom of Information Act to access all studies submitted to the FDA to get approval for 12 antidepressants that were on the market at the time.

What he wanted to see is what the FDA decision was on these studies, in terms of them being positive, negative or whatnot, and what then it looked like when these studies were published in the literature. Comparing what the FDA has around these pivotal trials, and how it looks like in the literature.

They found that there were, I think about 74 pivotal trials for these 12 antidepressants. About half, the FDA considered positive. About half, they considered negative. Actually, negative or possibly failed trials on their [inaudible 10:59], but let's put them together. Half positive and half negative.

Then, they went through what the literature was in these same studies. Who you'd expect if there's publication bias. You'd expect that some of these negative studies would be published as positive studies, and some of them would not be published at all. In the published literature for these trials, there were 48 positive studies and three negative studies published.

It is very striking. What's published in the literature is what guides us as clinicians. It's what guides clinical practice guidelines, and all such. The reality is, basically, what the FDA had. Another example I'll give you. The atypical antipsychotics at the time, the so called atypical, I should say.

The three that dominated the market in the mid 2000s were olanzapine, quetiapine and risperidone. They looked at all head to head trials between these three drugs that dominated the market. They found that whoever was funding the study, 90 percent of the time, their drug came out on top. You cannot rely on published papers. It's as simple as that, unfortunately.

You mentioned Gordon Guyatt before we started. When I started teaching on this topic, I didn't want to end with a nihilistic note. I wanted something to give my audience, something practical that they can take with them, so I started emailing a whole bunch of people.

I emailed Dave Sackett. I emailed Gordon Guyatt. I emailed a lot of people, saying, "You know, you guys are among the pioneers of EBM." Asking them, "Given the state that you acknowledge"    because these people acknowledge, it's pretty problematic "what suggestions do you have?" They didn't have very many practical suggestions for me, to be honest.

In fact, one of the responses I got was from John Abramson. He's with the Department of Family Medicine at the Harvard, in Boston. His response was just very nihilistic. He said, "We have to acknowledge that our evidence is completely screwed up. This is a bitter pill to swallow, but we have no choice but to recognize that."

I still don't stop there. I think that we have to make decisions as clinicians. There's ways to make decisions, but we have to acknowledge this very uncomfortable position that we're in today, that there's a huge schism between the actual evidence and the practice. The evidence itself, it's not easy to sort through, because the degree of bias in it is still huge.

Dr. Meuser:  What's your fondest hope?

Dr. Abi Jaoude:  My fondest hope is that we can do something that is very difficult to do as clinicians. It sounds obvious, but we acknowledge this, first of all, to ourselves. Not become nihilistic as a result of it, but be able to tolerate uncertainty and ambiguity. Again, it's not just psychiatry, but it's across the board.

We are increasingly recognizing that this is an important skill. The content is changing all the time, but there's some basics that I think are important. There was an article not long ago, in "The New England Journal." The title was, "Tolerating Uncertainty," where they're saying, "This is something we need to emphasize more in medical training."

As Stephen Hawking said, "The enemy of knowledge is not ignorance, it's the illusion of knowledge." Because if you're aware of the limitations of your knowledge, at least you're open to the possibilities. When you're able to tolerate doubts, you're able to be on your toes and think in different ways, and be open to alternatives. That as a first step.

Then, I think we need to go back to the roots, in terms of how we work with patients. The most effective thing that we have with patients, in general, is the therapeutic relationship. It's having a connection with the person, especially when it comes to primary care. Having an ongoing longitudinal relationship.

Everything we do, the degree of its success, regardless of the intervention, is very much dependent on the strength of that relationship. Then, when it comes to actual intervention, I think we have to be very mindful. Not rushed to what's ever in vogue these days, and following guidelines like technicians. Sometimes it involves going against the grain.

That's also an uncomfortable position to be in. I practice that way, and I'm very mindful that if something goes wrong I may be questioned. I'm going to be scrutinized. I might be [inaudible 14:56] off the college. I need to be ready to rationalize and defend it, and explain why I thought that what I was doing was truly in the best interest of my patient.

To think of a bigger context, a bigger picture, when you're talking about diagnosis. Then, when it comes to intervention, to practice conservative prescribing. There's a beautiful article written now, I think 2011. The first author is Gordon Schiff. He's an internist in Boston. The title is "Principles of Conservative Prescribing." It's really rational prescribing.

As you read through it, it includes things that are obvious, such as, "Beyond drugs. Don't jump into drugs right away." "Strategic prescribing. You don't just give the person multiple drugs like that." It also includes things that we may not think about as much, like being skeptical of the new stuff.

Five different people have recommended five to seven years before a drug is on the market, before starting to use it, because whenever a drug is new it seems promising. It's the best thing ever. That's like what William Osler said about new drugs. He said something like, "When a new drug comes out, you should use it as much as you can and as fast as you can before it stops working."

Waiting for a certain amount of time. Not jumping to new drugs freely. There are rarely true advances, when you come to think about it. Different groups have evaluated this in different ways, where they look at new drugs on the market in any given year. You look at what proportion was actually a true advance.

When we're talking true advance, it's a very small proportion, because it's hard. It's much easier and safer to develop a "#MeToo" drug. Practicing these principals of conservative prescribing, thinking of the big picture, these would be my hope, and being skeptical, overall.

Christine:  Given the substantial uncertainty and doubt about the quality of evidence clinicians are facing, not only in prescribing and practicing psychiatric care, but in medicine in general. Can you comment on what tools you might suggest for clinicians to utilize, to help advise their decision and the choice that they make?

Dr. Meuser:  Yeah, absolutely. First of all, when people hear about the literature, it feels overwhelming right away. I would say there's no need to be overwhelmed. There's no need to keep on top of the literature. Most of it is crap, anyway. There's the primary literature, but there's secondary literature.

I acknowledge I don't read the primary literature regularly. Then, if you decide that it's worth your while, you have to invest a lot of time and effort into it. You can't just take it at face value. You certainly can't just read the abstract. That's the most mindless part of the paper, in fact. You want to be very skeptical when you're reading it.

There's the typical classic Cochrane style of critical appraise. You want to do that, but even that's not enough. You want to be thinking broadly. You want to be thinking, first of all, who's funding the study. You want to look at clinical trials registries to see if the outcome measures have changed, from what was registered to what is published in the paper, because, invariably, they have been.

Just the narrative. Get a feel. "First of all, is this even a valuable question? Who's going to benefit from this? Did I get a feel that someone is trying to sell me something?" It does take a lot of energy to read through a paper. It's very doable, but I think you need to do it in a way that you're not just taking at face value what you're reading.

If you decide to read primary literature, that's ideal. If you don't, and most of the time you're not reading primary literature regularly, you're relying more on secondary literature, which I do.

Then the question becomes, "What secondary literature?" Because I would say most of these commercial entities are highly biased, partly because sometimes the free ones, which I can name a few, are highly biased. These are either entirely or in part related to what I refer to as max MECC, which stands for Medical Education and Communications Companies.

These companies have very interesting Web pages. I've been collecting these over time, where one side talks about improving healthcare, including the well being of physicians, and whatnot. The other side is marketing themselves to drug companies, talking about leveraging, influencing prescribers, etc. These are the free ones.

Some of them are very well known, and occasionally will have an interesting article. Even the non free ones, I would say it's hit and miss to what extent I would consider their work truly independent. Now, there are some that I would say are truly independent, and these are the ones that I follow.

Maybe I'll start with a Canadian one. There's a Canadian one that's free. It's called "Therapeutics Initiative." It's based out of the University of British Columbia. They're an excellent group. Unfortunately, they're not a very large group. Their funding has been repeatedly cut with pressure from pharma.

Some people criticize them as being overly critical and overly conservative. On the other hand, I think it's worth remembering that it is thanks to Therapeutics Initiative that [inaudible 19:34] never made it to the formulary NBC. That's their approach, and I trust what they have.

The others that I subscribe to, one is called "Prescrire International." Prescrire is a group based in France. They have a French publication, but they have one called Prescrire International, which is in English. They're very independent. They don't manage conflicts of interest, they just don't accept any conflicts of interest. They do things like...it's not just reviewing the literature.

They do very much the big picture stuff, which is, they go the gray literature, to the drug regulatory agencies. They go to the FDA. They get all the submissions to the FDA, but also in Europe, to the EMA, and across the world. They collect surveillance data from across the world, and all of that.

Then, the third one is called "Worst Pills, Best Pills." This is by Public Citizen in the United States. It's also subscription based. That one is very cheap, I should say. It's like $15 a year. The target is patients, but they give you the same kind of high quality analysis that Prescrire and TI does, for that matter.

Interestingly, all three of them pretty much align in their recommendations. In terms of the things that I've seen, they tend to align.

Dr. Meuser:  You've spent a lot of your energy and time looking at the difficulties with the application of evidence to clinical decision making in psychiatry. We can extend lots of what you say to other branches of medicine.

I'm sure you've had occasion to try to look for some light at the end of that tunnel. I wonder if you can talk about any opportunities you see for improving access and use of evidence for guiding clinical care in the future.

Dr. Abi Jaoude:  There's some light at the end of the tunnel, I just don't know how long this tunnel is. I don't know how stable it will be. I would say the major development is in the form of the recognition. That everything I'm saying today, if I were to say it 10, 15 years ago, certainly, I think I would have been considered a complete nutcase.

Today, people are saying these things, and they're not considered nutcases. That was why clinical trials registries were developed, for example, as a way to try to address that. Unfortunately, the clinical trials registries have failed in terms of their primary purpose because, first of all, the International Committee of Medical Journal Editors still published unregistered trials.

Second of all, there are journals that are not part of this group and will easily publish an unregistered trial. Third of all, that study is negative and it's not published, so who cares that it was registered? Fourth, most commonly is you change the outcomes and you publish it. No one is checking the outcomes. There are not enough people that are checking the outcomes.

It's failed in this respect. It's still useful if you access it regularly, and I...but this was an example of recognition. Another example of recognition is the whole discourse on transparency and conflicts of interest. We talk about this a lot now. We have to fill out yearly forms. When you're submitting a paper, there's a whole thing about these disclosures.

Again, this has failed in terms of its purpose, because who cares? The other thing is, declaration on its own is not enough. In fact, there's some behavioral psychology evidence that's interesting, suggesting it makes things worse, because the person making the disclosure feels they have a moral license now, so they can exaggerate even more.

On the receiver end, if they've heard the disclosure, they also let their guard down a little bit because they feel like, "OK, the person is transparently disclosing." The other thing is we need full access to the data. Not that we're all going to go through thousands and thousands of pages of data of every trial.

We need that data to be available for independent researchers to be able to interrogate it whenever they want. There's been a number of movements along these lines, around the world, that are promising. First of all, the FDA. I wouldn't say there's been particularly progressive moves in this regard, and nor would I expect anything very radical of the FDA under the current administration in the US.

Quite the contrary, unfortunately. However, having said that, they do make very thorough reviews available on their website for all trials. That's something that's a very underused resource.

In Europe, the European Medicines Agency, they've moved to making clinical study reports accessible publicly, as well. That's taken years because they've had challenges from drug companies that have challenged them legally. At this point, as we speak, CSRs are available through the EMA, and that's very valuable.

Then, Health Canada. Health Canada, we have some promising initiatives in Canada. You guys have heard of Vanessa's Law. I forget what bill number it is. This was brought forward by Terence Young when he was with the Conservative Federal MP. It's named after his daughter, who died after taking Prepulsid. She had sudden cardiac death from QT prolongation.

The company had interfered or had influence, I should say, with Health Canada. Vanessa's Law includes a number of things, one of which is access to data. It also gives Health Canada and the Minister of Health more power in terms of being able to remove a drug from the market or issue warnings and such.

Vanessa's Law has the potential to allow Canada to be the most progressive in terms of access to data in the world, but it all depends. The devil is in the details. From a systemic point of view, I see this as very promising, but also very cautiously so, because, as you know, companies with vested interests are watching these things very carefully and trying to influence things.

That's in terms of the what's happening systemically. On an individual practitioner level, I do find it promising that there's more awareness about this, and in particular, trainees. I see our trainees today as very bright. They tend to be progressive.

Again, it's one thing to interact with them on one on one, or as a group in a teaching session. To what extent they will be influenced by the establishment as they go through their training, or to what extent they'll be able to change things, remains to be seen.

[background music]

Dr. Meuser:  Thanks for listening to this episode of Clinically Speaking. It was produced and edited by Pippy Scott Meuser at the Center for Effective Practice.

Christine:  Special thanks to our guest this episode, Elia Abi Jaude, for taking the time out of his busy schedule to talk with us.

Dr. Meuser:  We're always looking for interested and engaged primary care providers to contribute to our development process. Interested in becoming a clinical lead or a working group member? Please visit our website at cep.health/participate to sign up.

Christine:  Don't forget to subscribe on iTunes, our website, or wherever else you get your podcasts.