On today’s No Stupid Questions podcast, there was mention of a study on the value of psychotherapy where the study authors (not the podcasters), stated psychotherapy had no value, and arrogantly added, therefore, all the studies that showed psychotherapy had value were now in question. I’ll address that arrogance later, but in the meanwhile, I’ll address the problem with that study’s bias that the podcasters, economist Stephen Dubner and psychologist, Angela Duckworth, didn’t fully reconcile, to my surprise.

The study was called The Comparative Impact of Cash Transfers and a Psychotherapy Program on Psychological and Economic Well-being, by Johannes Haushofer, Robert Mudida & Jeremy P. Shapiro. Basically, among 5,756 individuals in rural Kenya, the authors studied the economic and psychological effects of:

• A USD 1076 PPP unconditional cash transfer (i.e. gave some individuals money without condition);
• A five-week psychotherapy program (i.e. put some individuals through a standard World Health Organization psychotherapy program); and
• The combination of both interventions (i.e. gave some individuals both).

The authors’ conclusions? Improvements in economic and psychological well-being among people who got money, about the same with those who got both and psychotherapy, and basically nothing discernible for those who only got psychotherapy. Basically, the authors were saying the psychotherapy had no value and that it was the money that made all the difference.

On the data interpretation as if a dumb computer? Agreed. On the data quality, externalities, and data interpretation for an economist or scientist? Miserable fail. Of course, I can’t say that without defending it unless I were a troll, and when I looked in the mirror this morning, I was pretty human, so let’s start taking apart this nonsense by these authors.

First, the linked summary does not mention that these people were rather poor, which was mentioned in the podcast. This is the bias in the data that the authors failed miserably to compensate for in their data interpretation. It makes all the difference!

The reason the subjects’ poor economic standing is so vital is because of how money has a different impact on them than, say, compared to the entire population. Money will “buy” you “well-being” (sometimes called “happiness” to a certain point from zero disposable income. That point is where you can cover for all of your basic needs without worry, with a buffer for some general luxuries. While there is no Kenyan value, its American equivalent (for US citizens not monetary exchange) is beyond what these people would have been given. In other words, the money given to them would have been a guaranteed improvement to their lives to have some money to cover for at least some basics with a little more comfort than without the money, relieving some stress and bringing some joy, no doubt. In other words, the authors had set up a guaranteed success for themselves.

On the other hand, psychotherapy is not meant for resolving poverty, as far as I know. At least its most effective areas don’t include poverty. You can feel as relaxed as you want from psychotherapy, you’re still going to pay your basic bills and deal with consequences if you don’t. With regards to money, I would say psychotherapy can help you stop spending yourself into poverty, but it’s not going to help you earn income out of poverty. In other words, these authors had set up a guaranteed failure for comparison.

Put the guaranteed success and guaranteed failure outcomes together for comparison, and you get the guaranteed conclusion the study authors claimed they “found”. There’s no revelation there. The only revelation to be found is the subtle flaw of their methodology. So the only question left is did the authors know this, or were they blind to it?

Considering the way the authors were so arrogant to claim their study alone brought into question decades of other studies on the value of psychotherapy, I would say “yes”, they did know. This was a ploy, not some accidental claim from innocence of ignorance. These authors are pretty smart, overconfidently smart, I would argue, to know what I stated here but assume others would not know. So as for that arrogance, that’s deplorable. It’s using the age old strategy of claiming or doing something controversial just to garner attention for yourself. Well, I regret to inform them they also failed at this… miserably failed.

 

721  words