Book Review: Paradoxes of Rationality and Cooperation

What is rational choice? Paradoxes of Rationality and Cooperation (1985) is an anthology of papers that examine this question through the lens of two classic thought experiments: the Prisoner’s Dilemma and Newcomb’s Problem. Both of these decision problems present challenges for conventional notions of rationality and raise difficult questions about the role of causality in agent reasoning. Beyond the problems themselves, the various authors consider conflicting philosophies of rationality and the effectiveness of their corresponding decision strategies.

The volume traces the history of academic thought (and disagreement) on the choices within the two problems during the 70s and early 80s, with several chapters responding directly to critiques from other included authors. While this discussion is sometimes heady, Paradoxes highlights the applicability of these ideas alongside the theory (such as in the final section on iterated Prisoner’s Dilemmas) and grounds the anthology in that context:

Quite simply, these paradoxes cast in doubt our understanding of rationality and, in the case of the Prisoner’s Dilemma, suggest that it is impossible for rational creatures to cooperate. Thus, they bear directly on fundamental issues in ethics and political philosophy and threaten the foundations of the social sciences. It is the scope of these consequences that explains why these paradoxes have drawn so much attention and why they command a central place in philosophical discussion (Campbell, 3).

The Prisoner’s Dilemma

Briefly, the Prisoner’s Dilemma presents a symmetrical decision problem where you and another prisoner must each decide, in a completely isolated way, whether or not to betray each other (typically this is presented as either confessing or remaining silent). Each person’s imprisonment is determined by the resulting decisions and the following matrix, applied from their perspective:

Regardless of what the other prisoner does, your outcome is better if you choose to confess (we would say that confessing ‘strongly dominates’ not confessing). Because the situation is symmetrical, the other prisoner has the same incentive to confess and so a seemingly-rational approach leads to both prisoner’s confessing and receiving nine years when they both could have stayed silent and received one.

The specific lengths of the sentences are irrelevant as long as the order of preferred outcomes for each prisoner is the same. Some motivating examples are provided in the book’s lengthy introduction:

Imagine that instead of zero, one, nine, and ten years the jail terms are none, one day, forty years, and forty-five years respectively. The same reasoning proves that each prisoner, if rational, should confess so that both spend forty years in jail instead of a single day. Or, to make the contrast even more bizarre, suppose that the first two possibilities are freedom plus $10,000 and freedom plus $1,00 while the second two are a quick, but painful, death and slow death by torture (Campbell, 6).

The ‘paradox’ of the Prisoner’s Dilemma is that it seems to easily extract poor outcomes from supposedly-rational agents when superior outcomes appear readily available. Unless we are prepared to accept these limitations, it seems necessary to expand our notion of rationality beyond intuitive, dominance-based reasoning.

Several authors advance a ‘symmetry’ argument that seems to allow for cooperation among rational agents. First, consider the proposition that there is a unique rational choice in the Prisoner’s Dilemma. A proof by contradiction: 1) if there is no unique rational choice than neither agent can be sure what the other will do; 2) as a result, the only rational choice is to confess, since this results in a superior outcome either way; 3) but that would mean there is a unique rational choice, contradicting the original assumption.

Having (possibly) convinced ourselves that there is a unique rational choice, we analyze the agents’ decisions. If each agent is rational and knows the other to be rational (as is typically supposed), they can reason that if confessing were rational then they and the other would confess. Likewise, if silence were rational then they and the other would remain silent. In other words, there are only two possible outcomes: both confess or both remain silent. If only these two outcomes are possible, then remaining silent dominates confessing and both agents will remain silent.

This argument is not entirely convincing on its face and several of the papers argue its merits from either side. In particular, debate arises from the assertion that mixed confess/silent results are not ‘possible’. To illustrate the epistemic sense of possibility (and hopefully improve the argument’s standing), consider this justification from one advocate: imagine an additional option in the Dilemma called “confess+” which adds to each’s sentence regardless of the result. The five related results in the (now) 3x3 decision matrix are not ‘possible’ in the sense that rational agents will obviously not choose them. The cooperation argument asserts that there is exactly one ‘possible’ result: the result which will occur (given the proof that there is a unique rational option).

Another critique of the symmetry argument is that it seems to imply a causal relationship between the decision of the two agents where none exists. However, several papers point out that agents’ behavior can be highly correlated without a direct causal relationship between their choices. If the agents’ rational nature (as a third factor) is sufficient to make it likely (though not certain) that the agents will make the same choice, then the principle of maximization of expected utility will recommend cooperation.

If an argument for cooperation cannot find footing then the Prisoner’s Dilemma casts doubt on the possibility of cooperation in analogous situations where other remedies like precommitment or third-party enforcement are not available. The general form of the problem clarifies how this dynamic is applicable to familiar, real-life scenarios:

The parties to the choice must have partly overlapping and partly divergent preferences regarding the possible outcomes. Specifically, both must agree in preferring one of the four possibilities to another but disagree about which is best and which is worst. Moreover, neither can force the other to make one choice rather than the other. So achieving the next best outcome for each, rather than the next worst, demands cooperation and trust. It is hard to imagine a marriage break-up which would not meet these minimum constraints, perhaps in more than one way. And so it is for countless other situations in which parties to a dispute must cooperate at some risk in order to take advantage of the common ground between them (Campbell, 7).

As we will see in the final section, an iterated version of the problem is required to clearly justify this ‘cooperation under risk’.

Newcomb’s Problem

Though less realistic than the Prisoner’s Dilemma, Newcomb’s Problem drives more directly at the tension between dominance reasoning and expected utility reasoning. One presentation is as follows:

Newcomb’s Problem concerns a being with enormous predictive powers. You have overwhelming confidence in these powers; he has already correctly predicted your own choices in numerous situations and the choices of many others in the situation you now face. There are two boxes before you: box 1 contains $1,000 and box 2 contains either $1 million or nothing. You have two choices: to take the contents of both boxes or to take the contents of box 2 only. You know that the contents of box 2 depend on the being’s prediction, in the following way. If he predicted that you will choose both boxes, then he put nothing in box 2; and if he predicted that you will choose only box 2, then he put $1 million in box 2 (Horgan, 159).

Each option has an intuitive appeal. On one hand, the being has already made its prediction and placed (or not placed) the million dollars in box 2. No aspect of your choice can alter the state of the boxes so you have nothing to lose by taking both boxes. On the other hand, you know empirically that those who take only one box almost always find the million dollars while those who take both almost always find box 2 empty. The choice is polarizing, as one author reports:

I have put this problem to a large number of people, both friends and students in class. To almost everyone it is perfectly clear and obvious what should be done. The difficulty is that these people seem to divide almost evenly on the problem, with large numbers thinking that the opposing half is just being silly (Nozick, 110).

As with the Prisoner’s Dilemma, authors in Paradoxes fall on either side of the question and argue back and forth. The central conflict is around the method of expected utility calculation. The ‘evidential decision theory’ (one box) position is that a simple expected value calculation justifies taking one box. The ‘causal decision theory’ (two box) retort is that only causal probabilistic relationships should be accounted for in the expected value. It is not possible to ‘manifest’ the million dollars by choosing one box so taking both strongly dominates.

In an effort to make progress on the problem, several authors discuss modifications that make the choice clearer. If the ‘predictor’ is known to have reduced accuracy (say, 60% historically) then the expected value of taking both boxes becomes greater than taking only one if money has linear utility. This raises a question for one-boxers: what is the minimum predictor accuracy needed to justify taking only one box?

Likewise, if we say that the predictor is perfectly accurate then one-boxing is the clear best option (though this situation raises some questions about the nature of the predictor). Two-box advocates then face their own dilemma: what justifies a change in strategy from one-boxing to two-boxing when the predictor’s accuracy drops (for example) by one part in a billion?

Beyond this point the discussion gets murkier and less satisfying. Various authors try to advance the standing of their preferred decision theory with weak results. Most papers skirt more fundamental questions about the nature of choice: to what extent is a decision a computation by the mind of the decider? How do we reconcile (if at all) our idea of free will with concepts like a perfect predictor or (as in other popular problems) a genetic predisposition for certain behaviors like smoking? While Newcomb’s Problem itself is a wonderful brain teaser, the academic positions within Paradoxes offer little beyond entertainment.

Iterated Prisoner’s Dilemmas

The most applicable content of the book is the final section which examines ‘large-scale’ iterated Prisoner’s Dilemmas where many agents participate in repeated, pairwise, cooperate/defect choices with a typical Prisoner’s Dilemma payoff structure while knowing the identities of their counterparts. Simulations of various decision strategies in these situations provide insights applicable to international and domestic political economies as well as the pedestrian interactions of day-to-day life. Of particular concern are the conditions that motivate cooperation in a population of independent, self-interested agents.

Pairwise simulations of definite length are the most straightforward: both agents are incentivized to defect on the final move of the simulation since their counterpart will have no chance for reprisal. However, both know this and are therefore incentivized to also defect on the penultimate move, and so on. More interesting are simulations of indeterminate length where mutual cooperation can allow agents with ‘nice’ (i.e. initially-cooperative) strategies to outstrip defectors. In fact, simple strategies like ‘tit-for-tat’ (where an agent cooperates on first interaction and then parrots a counterpart’s previous action in future moves) can be successful by allowing repeated mutual cooperation even while paying the cost of initial defection by other agents.

One author describes a tournament of various decision strategies where a large number of pairwise encounters between various strategies were simulated. Tit-for-tat fared best among many approaches: “Tit-for-tat was a very robust rule because it was nice, provocable into a retaliation by a defection of the other, and yet forgiving after it took its one retaliation” (Axelrod, 326). Tit-for-tat also performed well as the ‘field’ of decision strategies became more competitive:

To see if tit-for-tat would do well in a whole series of simulated tournaments, I calculated what would happen if each of the strategies in the second round were submitted to a hypothetical next round in proportion to its success in the previous round. This process was then repeated to generate the time path of the distribution of strategies. The results showed that as the less-successful rules were displaced, tit-for-tat continued to do well with the rules which initially scored near the top (Axelrod, 326).

One important formal aspect of a decision strategy is stability. A strategy is said to be ‘collectively stable’ if no other strategy can outperform it as a lone outlier in an otherwise homogenous population. If a single actor using strategy B can achieve a better result over many iterations in a population of actors using strategy A, then B is said to ‘invade’ A. Tit-for-tat is proved to be collectively stable as long as the probability of additional interactions is sufficiently high, somewhat formalizing its robustness. However, as the probability of additional interactions is lowered, the strategies of ‘alternate cooperate/defect’ and ‘always defect’ are able to invade. This dynamic is readily observable in human interactions, as one author notes: “It is easy to maintain the norms of reciprocity in a stable small town or ethnic neighborhood. Conversely, a visiting professor is likely to receive poor treatment by other faculty members compared to the way these same people treat their regular colleagues” (Axelrod, 330).

This property of tit-for-tat, that it is only collectively stable when additional interactions are somewhat likely, is formalized mathematically in chapter 19 alongside other generalized properties of collectively stable strategies. For example, any strategy that cooperates must be provokable by the first defection of a counterpart in order to be stable. The prospect of future interactions is also shown to be a necessary prerequisite for cooperation in the general case, as it is with tit-for-tat. Finally, it is shown that the strategy of ‘always defect’ cannot be invaded by any strategy, since a lone agent with a ‘nice’ strategy has no one to cooperate with. However, this only holds for a single invader.

If, instead, a ‘cluster’ of newcomers is introduced, the weakness of ‘always defect’ becomes apparent. As long as a tit-for-tat agent has some small percentage of its interactions with like-minded cooperators, the cooperating agents will outperform the defector majority. For example, if the payoff structure is 5/5 for mutual cooperation, 5/0 (or 0/5) for mixed responses, and 1/1 for mutual defection, and if each interaction has a 10% chance that the two agents will never meet again, then a cluster of tit-for-tat agents need only 5% of their interactions to be with other cluster members to outcompete the defectors.

‘Nice’ strategies that initially cooperate (like tit-for-tat) do not have this weakness; if a ‘nice’ strategy cannot be invaded by a single agent with another strategy then it cannot be invaded by a cluster either. “So, mutual cooperation can emerge in a world of egoists without central control, by starting with a cluster of individuals who rely on reciprocity” (Axelrod, 338).

This is a heartening result. Cooperation and trust are often the heart of our lives as social beings. Pulling apart the pieces of rationality, it is pleasing to find that altruism and self-interest can be complementary forces.