Being a Psychiatrist you may assume I am naturally going to be biased favorably. However, you may be surprised to learn that is not always the case. A Meta-Analysis published in Molecular Psychiatry took a very thorough look at this matter and here I am presenting a quick critical review of this study  and its findings.
The main author of this study is Stefan Leuchat who is a Professor at the Psychiatry & Psychotherapy Department, Technical University of Munich, Germany. All the other study authors are also German based except one JM Davis who is based in Chicago, IL, USA.
Conflicts of Interest
As an overview, it seems their university department is both Psychiatry and Psychotherapy oriented, which should bridge some gaps (hopefully) between the two approaches.
The authors also explicitly state that this meta-analysis did not receive any funding. However, Stefan
Leucht has received speaker or consultancy honoraria (meaning being paid to speak or consult) from Sanofi-Aventis, BMS, Lilly, Janssen, Lundbeck and Pfizer. And that Lilly and Sanofi-Aventis sponsored some research projects done by Dr. Leucht. Another author Werner Kissling has also received speaker and consultancy honoraria from many of these ‘Big Pharma‘ corporations.
Some critics and recent scientific studies have challenged the usefulness of medications used by Psychiatrists. One of the studies found that anti-dementia drugs are not significantly effective and in fact, cause side effects and result in costs to the system as well as suffering for the patients .
Another one challenged the usefulness of Anti-depressants finding that they only made a very small difference to the symptoms as compared to a placebo .
In this context the authors conducted this meta-analysis to find out if Antipsychotics were useful.
Study Design and Method
The breadth and depth of their search for relevant studies seems to be impressive and done thoroughly. The authors mainly used a large database that indexes studies from multiple other databases in English and other languages, it also catalogs an extensive amount of gray literature. This database is maintained by the Cochrane Schizophrenia Group.
They do try to stick to quality studies by having robust inclusion criteriums i.e. only Randomised Placebo Controlled studies were included which met certain quality criteria. They also did some sensitivity analyses to check for the robustness of the results.
One cannot fault the method and statistical analysis used much. Except it is unclear how they ended up with the 38 studies (that they analyzed) as it seems they had numerous citations after running the searches. But they then go on to say “Of those publications that we ordered for inspection..” yet it unclear how they decided on which ones to order for inspection, they ended up excluding 107 (for design issues) even from those they ordered for inspection.
Interestingly all the final studies were conducted by (you guessed it) pharmaceutical companies.
There are some very interesting Results:
This graph is the most interesting result for me. As you can see it appears the size of effect Antipsychotics (newer) seem to have on reducing the symptoms of psychosis as compared to a placebo (i.e. fake drug) has been constantly reducing with time. In around 1983 a small study showed an enormous difference in effect size of >-1.6 (0.8 is considered to be large), whereas in 2007 a large study is showing the difference of effect size to be merely -0.2. This is a huge difference and the trend is statistically significant as well.
So how do we account for this result?
The author’s do not appear to give any satisfactory explanation for these results. They note that the placebo response rate seems to be high and also that a large number of dropouts (around 47%) overall in the studies suggests any difference that might have been observed is reduced.
However being more cynical we need to stop and consider several other possibilities here:
- How much of these ‘scientific’ results have been influenced by the money politics of big pharma?
- If placebos can also substantially treat psychosis then whats the use of antipsychotics, especially when the difference between them seems to be ever decreasing?
- How does this then reflect on all the other schizophrenia research being conducted? where dopamine overactivity is the most favorable hypothesis so far (yet anti-dopamine drugs i.e. antipsychotics seem to work less and less now)
There is also significant publication bias:
As you can see there seem to be a few lower quality studies (i.e. with a higher standard error) with positive results. However studies which should be expected within the red circle are missing. Studies with a higher standard error should naturally have a wider variation in results so one would statistically expect there to be negative studies as well (i.e. within the red circle), yet they seem to be missing.
This is interesting because most of the studies that were published were done by pharmaceutical companies, from there we can only assume they likely did not publish the negative ones. The authors, although agree that there is a significant publication bias and agree with the above possibility, they also suggest that it could likely be due to the heterogeneous nature of the sample studies.
So do Antipsychotics work?
Having considered the above we still need to be cautious in making premature judgments. The pooled size of the effect for antipsychotics seems to be -0.51, meaning it’s a moderate effect. This is equivalent to a minimally visible difference between two things when seen by the naked eye. The caveat here is the trend of a decreasing effect. Another caveat is that the effect size for even a fake drug is substantial in comparison and in fact, the difference in effect between the two only amounts to 18%. This means 6 people will need to be treated for 1 more person to benefit on an antipsychotic as compared to a placebo drug.
Perhaps they are better in relapse prevention then? It seems so, however again the difference from placebo is only 20%.
Furthermore in 2 studies one of the antipsychotics i.e. Olanzapine appears to improve the overall quality of life for patients.
Overall the authors rightfully point out the previous studies may have overestimated the effectiveness of antipsychotics perhaps because they were not very pragmatic in terms of what type of patients they included in the study.
What about their nasty side effects?
On a more positive note, contrary to what the general perception is among the public and professionals the newer antipsychotics don’t seem to cause any significant side effects compared to a placebo medication. In fact, few of antipsychotics led to fewer people dropping out (for any reason) compared to placebos. The older antipsychotic (haloperidol) did show significantly more side effects.
It doesn’t appear that antipsychotics are immensely useful drugs especially given their apparent reducing effect size as time goes on. However, the newer ones don’t seem to be significantly harmful either. It seems even minor effects may still be worthwhile given we don’t have any better drugs for the time being.
This study, in a more general sense, also reflects on the scientific enterprise as a whole. It goes to show that having scientific evidence in favor of some position doesn’t always amount to absolute truths and that results can vary greatly from time to time. This could in part be because of further developments in methodology or because of the nature of studying human sciences. One also has to wonder about the more cynical reasons in getting widely different results i.e. political, monetary etc.
Do share your thoughts below.
- Leucht, S., Arbter, D., Engel, R. R., Kissling, W., & Davis, J. M. (2008). How effective are second-generation antipsychotic drugs? A meta-analysis of placebo-controlled trials. Molecular Psychiatry, 14(4), 429-447. doi:10.1038/sj.mp.4002136
- Bentham P, Gray R, Raftery J, Hills R, Sellwood E, Courtney C et al. Long-term donepezil treatment in 565 patients with Alzheimer’s disease (AD2000): Randomised double-blind trial. Lancet 2004; 363: 2105–2115.
- Moncrieff J, Kirsch I. Efficacy of antidepressants in adults. BMJ 2005; 331: 155–157.