Lessons from Monty Hall

I wasn't the first to get the Monty Hall problem unequivocally and humiliatingly wrong. But I was as incredulous and arrogant as anyone else.

Monty Hall, you may recall, hosted a game show where contestants had to guess the location of prizes  behind a number of closed doors. In the classic Monty Hall problem there were three doors, two with nothing behind them and one with a grand prize. After the contestant chose one of the doors Monty Hall advised which of the other two did not have the prize. There were now only two doors where the prize could be located -- the one that the contestant had chosen and the one of the other two that Monty Hall did not rule out. Monty Hall then gave the contestant the opportunity to change her choice.

The question is: was there any difference in the probability that the prize was behind either one of these two remaining doors.

I, like many others, insisted there was no difference. The facts were clear. After the information Monty Hall had provided, ruling out one door,  there were only two doors where the prize could be located. The chance that the prize was behind either one of them was the same -- one in two.

That still makes perfect sense to me, except it is wrong. It ignores the fact that from the outset when the contestant picked one of the three doors there was a two in three chance she was wrong. To say the same thing there was a two in three chance that the prize was located behind one of the two doors she did not choose. And that didn't change when Monty Hall ruled out one of the two non-chosen doors. The only change was that there was now only one non-chosen door with the two in three chance of having the prize behind it.

So if the contestant stayed with her original choice she would have a one in three chance of winning. But if she changed her choice to the other door she would have a two in three or double the chance of winning.

I know this is a bit confusing and counter-intuitive. My son, or as my wife would say, the brighter and nicer economist in the family, had to construct simple simulations of the game to convince me I was wrong. Pure logic wasn't enough. But after a while it sunk in. And it was a great learning experience -- not so much the mathematical probability part but rather the more fundamental and humbling recognition of how wrong we can be despite the certainty with which we hold to the correctness of our position.

It is, I think, one of the most important lessons that helped shape my economic consulting career.  No matter how carefully we do our analysis and how thoughtfully we develop our conclusions and recommendations we must never forget we may be wrong. We miss or ignore important facts. We fail to recognize  the nature and significance of our assumptions. We sometimes just make mistakes.

I've always been skeptical of those who profess they have the one and only correct answer to difficult public policy issues. That is why I have taught and applied a multiple account approach to the evaluation of policy and project alternatives. Different values and interests can bring different perspectives and often the best we can do is to assess the consequences of different actions -- how things would be different going forward -- in order to define as clearly and meaningfully as possible the nature of the trade-offs different alternatives present. Add to that a real appreciation that we may be wrong in our assessments despite all of our best efforts and it calls for some caution and humility in the work we do.

One of the most unfortunate trends I've witnessed over my consulting career is the exact opposite of caution and humility. Especially in contentious policy and project debates, and the hearings and media frenzy that often go along with them, there is far too much of what I like to call "analysis with attitude". People have a position and shape their assumptions, analysis and conclusions accordingly. There is no room for nuance, recognition and assessment of trade-offs, or simple acknowledgement of all the reasons including typically pervasive uncertainty let alone mistakes, as to why they may be wrong.

After over 45 years I am closing my consulting business. If I can offer anything from my experience it is a plea to those who continue in the very important world of public policy analysis, debate and advice -- stay away from analysis with attitude and the vitriol and intolerance it too often foments. Try to inform and elevate the debate with careful, objective analysis and a fulsome recognition of uncertainty, different perspectives and interests, and the simple fact that despite our experience, knowledge and best intentions there are lots of Monty Hall problems and paradoxes out there. We make mistakes.