Most things are made of some kind of group, your neighbourhood, your workplace, the UN. Yet there are many times when the behaviour of the group does not seem to conform to what we would expect. If we got a look at the members who made it up we may be surprised at what the outcome is. Why is this? Predominantly inspired by Schelling’s Micromotives and Macrobehaviours I’ve tried to lay out some systems that can help shed light on this.
Schelling in his book stresses early on that just because a system is in a particular equilibrium does not mean it is optimal or even desired by the participants, to paraphrase, even a hanged man will stop swinging. In many situations there are no mechanisms that relate individual actions to some collective goal and so we shouldn’t be surprised that many situations are not optimal or even representative of people.
Before we look at the systems themselves are there any good measures we can use to gauge to overall properties of some system? A paper by the Santa Fe Institute looks at collectivity and proposes two measures.
The first is amplification; how much will changing the components of a system have an effect on the aggregate behaviour. Positive feedback loops can have high levels of amplification as small changes in peoples beliefs get propagated forward by their behaviour. Negative feedback loops have much lower levels of amplification with small changes being dampened even further.
The second measure is decomposability; how much of the aggregate output can be decomposed into effects from distinct subgroups. Decomposability is essentially a measure of how different the group is from the individual, which is exactly what we are interested in. It mainly affects reflexive situations and aggregation situations. In reflexive situations your behaviour is partly determined by the behaviour of others and so you can’t look at individuals alone to tell you what is going to happen. In aggregation situations as the aggregate is made up of the members and can have properties no member can individually then looking at any subgroup can be misleading. If it still seems strange that there may be cases where you can’t look at the elements of a group individually to determine what will happen the paper also gives the simplest example I know of synergistic information, the XOR gate. Neither bit alone tells you any information but with both bits you have all the information.
I have broken the systems up roughly into two groups, preference enhancing and preference masking. These are not clear cut, there are reflexive situations that make individual preferences difficult to tell but I think it is a useful distinction to broadly keep in mind. Preference enhancing systems have the propensity to lead to strange results by dramatically accentuating the individuals preferences/beliefs. Preference masking systems can lead to seemingly strange situations by making the motives of the individuals very difficult to discern. The systems are constructed with the assumption that agents within them act in a logically consistent way i.e they take actions that are at least locally optimal, other motives are not explicitly addressed.
Preference enhancing
Reflexivity
There are many systems where participants both have beliefs/preferences about the system they are in and affect the system causing a feedback loop. This is by far the broadest category and so will be the longest section. As I’m sure you can tell, the description I’ve given above is vague, but intentionally so.
Reflexivity is about feedback loops. George Soros talks about humans having a cognitive function where we try understand the world and a participating function where we try to change the world to our advantage. The issue arises because the use of both functions at the same time deprive both of their independent variable. Taking actions changes the world in which you those actions are taken in. Trying to figure out the world can lead you to do things that change the world you are trying to figure out.
Another issue is that we are fallible, we always hold imperfect views on the true nature of things, and hence act in inappropriate ways. These actions change the world and as our view was imperfect they change the world in ways we weren’t expecting and so our views and actions will constantly change.
If we look at the feedback loop involved in reflexive systems more deeply we see that there are a few different ways they can operate. In simple cases there is one reflexive variable that people have some view on. In some cases people have a preference towards a variable that is always true, e.g how much the average person tips a waiter. Obviously there always has to be an average it is just that peoples behaviour can change this value. Another case is where people have a belief about some value. In this case, say the chairman of the Fed, for whatever reason, says the economy isn’t looking great, it is a belief on the truth of some statement. Now this statement need not have to be true or false before it was made but once it is made it may influence its own truth. If the chairman of the Fed says the economy is looking bad that might well make the economy bad. This trait is oddly human, we would be amazed if our predictions on the course of Halley’s comet changed its course, but our predictions on bank runs can change how people act.
Dynamics
Being in a reflexive system just means that you have a preference/belief about something that you can control and that your actions affect others who also have preferences/beliefs about the same thing. There are however multiple ways that the types of preferences/beliefs can lead to different types of group behaviour. I’m going to look at five different ways being in a reflexive system can play out over time.
I’ll start with the least time sensitive option the self defeating prophecy. In these situation individuals have a belief and the more people who hold such a belief the more likely it is that it will be false. For example, if everybody believes that a certain event will be too crowded and that they don’t want to go if there are too many people then we could be left with a situation in which very few people go and the event isn’t crowded at all. Another case could be the more people who believe the candidate they support is going to win by a landslide the fewer of them think their individual vote will matter, fewer of them turn out and their candidate ends up losing. Self defeating prophecies are typically one shot but if they are to do with process that will repeat then they will likely turn into a self correcting prophecy as people adjust to their mistakes.
The most well known ‘prophecy’ is the self fulfilling kind. This is where certain expectations induce behaviour that will cause those expectations to be fulfilled. Schelling goes further though and breaks this down into a large number of subgroups. Note that these subgroups all describe group to group beliefs as these are the original environment of the term self fulfilling prophecy.
Firstly there are unilateral processes where one group believes somethings about another, this leads them to act towards the other group in some way which in turn can lead to behaviour from that group that confirms the first groups beliefs. A classic example, and in fact the one Schelling references as being the first example, is whites believing black citizens were less capable of handling responsibility in a job setting. Thus they didn’t hire them and this lead to blacks to not develop the skills they would need in order to perform some tasks well thus making them in fact worse at those jobs.
The second type is reciprocal processes. Here two groups have beliefs about each other which leads to them acting in ways that cause the other group to conform to their expectations. One example of this could be Israelis and Palestinians. If both expect the other to be aggressive towards them, this may lead to them treating the other with distrust causing both to be aggressive. This situation lends itself to coordination misalignments that we will look at later. It may be that if both groups were less aggressive both would be better off but for either group there is always some benefit to being the slightly more aggressive actor.
A third type are selective processes. These occur when a group has some belief about something which is widely known leading to people changing their own behaviour to conform with these beliefs. One example could be if everyone expects teachers to be very bubbly people. It may be that this ins’t a necessary part of being a good teacher but people see this from the outside and self select into not applying to be a teacher if they don’t view themselves as bubbly.
If we move away from simply group on group there are two important subgroups.
The first type are critical mass systems. These are all or nothing affairs where being in the minority becomes more disadvantageous as the majority grows. For example, not withdrawing your money from a bank gets even worse as more people think there will be a bank run and act accordingly. The result will either be a bank run, if sufficiently many people believe one likely, or no bank run, there is no real in between. An interesting example of critical mass systems are self enforcing systems. These occur when once a certain amount of people have agreed to something there is no incentive to change from this behaviour. An example of this might be if everyone expects to drive on the right of the road, it could go either way in the decision process of which to pick but once there is some certain amount of people doing the same thing then it makes very little sense to change from that. These systems can be part of the dynamics that set up coordination misalignments that we will look at in a bit. One especially important thing to keep in mind with critical mass systems is that they mean that looking at the groups behaviour can be even less revealing about group preferences than other reflexive systems as very small changes can lead to dramatically different outcomes.
The second subgroup I will call non-critical mass systems. These are situations where the more something is expected the more it will happen, but it need not be an all or nothing affair. For example, the more people who think attendance will be poor the fewer people will show up. With a bank run it gets much worse to not do anything the more others believe there will be a bank run, but with the crowd example it doesn’t get much worse as fewer people show up, only somewhat worse.
A third kind of overarching dynamic is the self displacing prophecy. These occur when everyone sharing some belief/motivation and all basing their behaviour off of some value will systematically displace that value from where they thought it would be. For example, if everyone wants to tip slightly above average then the average value tipped will increase leading to people needing to tip even more to stay in line with their preferences. If that seems a little contrived consider universities. Each university may want to award a slightly higher number of students good grades than the average university. This by itself can make a lot of sense. If a university does this then they might assume the total number of people achieving some grade won’t change much but they will capture a larger percentage of those with that grade. But if every university does this then we suddenly get grade inflation, which is, oh, what we are seeing now. This also has a lot of links to coordination misalignments. It might well be optimal for each university to do this but it leads to suboptimal results in the aggregate as they are targeting a reflexive variable.
A fourth kind of prophecy is self correcting/self equilibrating also commonly known as negative feedback loops. Here, some original stimulus triggers a response that may still be more extreme than desired but is closer to the level one desires. There is a prediction about large scale environmental damage, this causes people to become much more environmentally conscious leading to the damage not occurring. It is then expected that the environment will become better much faster than expected. This may then lead to people increasing there pollution levels again but probably less so than originally etc. This is different from the others mentioned so far in an important way, beliefs won’t accelerate off indefinitely, it is important though to note that just because the equilibrium can be stable that does not mean it is optimal.
The fifth kind is different from the above in that it is the result of a combination of prophecy types. This is where a system is in dynamic disequilibrium. George Soros talks about this being the main type of system many human activities lead to, being behind cycles in things like financial markets. In these situations people are prone to positive feedback loops that reverse once they become too extreme. This leads to people over-reacting to events which continues until things become so far removed from reality that there comes a point where people realised the group is wrong and it becomes in their interest to change. This reverses the trend but again there is an overshoot. This is different from self correcting situations because the degree to which people over-react does not necessarily decrease over time, rather they oscillate indefinitely around some value.
Segregation example
Schelling gives what is probably one of the most vivid examples of a reflexive system in his segregation example. Here we start with a neighbourhood with #’s, O’s and some empty spots. The rules are simple if either type are surrounded by fewer than three of their own type they move to a spot where this isn’t true. Just to emphasise this means that either type is happy even if most of their neighbours are not like them, they just don’t want to be too heavily outnumbered. So what happens? If we start with the distribution in Fig 4 one potential outcome that we reach is Fig 8.
Fig 8 is far more polarised than we would expect given the individual preferences but this is no accident. When either type moves they are moving to an area where there are already a suitable number of the same type. Them moving makes this area even more homogenous. Similarly when they leave an area that had a paucity of their type they deprive the area even more of variety making others of their type more likely to leave. In this case you not only have preferences about your neighbours but are the neighbour of all those around you. If you want to play around with this model more the best resource is the Parable of the Polygons site.
Coordination misalignments
These are situations where actions that are locally rational to each individual lead to an outcome that is globally suboptimal e.g the prisoners dilemma.
Scott Alexander has a brilliant essay called Meditations On Moloch where he looks at these coordination style games. His central question one I’m sure many of us have considered when looking at some of the outcomes we see around us; If things in society are so bad and society is just made up of us, why don’t we change how we act and make society better? A noble thought but clearly something is stopping us. Schelling states as the central issue “How well each does for himself in adapting to his social environment is not the same thing as how satisfactory a social environment they collectively create for themselves”. If no dynamic is present to push us towards some globally optimal result why should we expect one?
Let’s start by looking at a slightly contrived example that vividly shows locally optimal actions leading to globally suboptimal ones, the dollar auction. For those not familiar the way it works is that I hold up a dollar, you and another person bid for it, the only catch is that whatever your last bid was you will have to pay regardless of whether you win or lose. If you open the bidding at 10 cents the next person has every reason to bid 20 cents, after all if they don’t then you’ve just won a dollar for 10 cents. Feel free to play this with someone else or just you acting as both parties to get a feel for what might happen. The results are pretty surprising, and one university professor has found that he can often get four times the amount people are bidding on. At each stage, except possibly agreeing to even play, it makes sense for you to up your bid, if you do you will pay 10 cents more, for example. For this 10 cents though you save yourself losing the entirety of your previous bid. The issue is that at every stage your sunk costs are much greater than the cost of becoming the leader
A less contrived example could be a global arms race. In this case it is always advantageous for you to have slightly more weapons than your enemies but collectively this leads to a situation where you all have a huge budget for defence, huge inventories of weapons and a huge desire for everyone to reduce their arsenals.
One of the most famous examples is the tragedy of the commons. You have some common pool of resources, say the fish in the ocean or some grass in a village. People can use this pool for their private benefit but by doing so it exerts some cost on all the other participants. Similar to the previous examples, ones locally optimal action, fishing more or letting more of you cattle graze, can lead to globally suboptimal results, no fish or no grass left. You can see this easily form the outside and yet for every participant in the system their incentive is to just keep going even though it may ruin them.
Meditations On Moloch contains a metaphor I find myself reflecting on a lot which is the ‘incentive landscape‘. Just as the course a river takes is there in the terrain even before any rain has fallen so to is how people will react to a situation even before they have encountered it. Every incentive or situation acts as a feature of a landscape and people obeying their incentives are like water flowing through it. This is not to say things can’t change. We’ve built canals and dams and likewise the incentive structure of institutions can be altered it is just that it will likely take some coordinated effort that individual agents may not be able to resolve.
Some of these problems have a reflexive aspect to them as well. You want more bombs than the average country so you stock up but therefore also raise the average number of bombs a country has. If other countries also have this preference then this can lead to a self displacing cycle. The reason I have split this into a separate category is that the dynamics of coordination problems don’t require any reflexive aspect to operate. In the bombs example you could have very slightly different problem with similar outcomes that wasn’t reflexive i.e you just want 10 bombs vs 5 even though you would all be better off if each country only had 5 bombs each.
Preference masking
Asymmetric info
Here differences in what groups know about each other leads to them acting in ways they would otherwise not should they know more. In these cases individual preferences/beliefs are often homogenised leading to behaviours that would not occur were all individuals able to be treated as uniquely.
Adverse selection
Here we have asymmetric information before some event. This means that one group cannot fully get across some piece of information about themselves that another group would be interested to know. For example, with used car sellers may not be able to fully indicate the quality of the vehicle they wish to sell (aka the market for lemons). This means that buyers would only be willing to pay the average asking price. This however means that any seller who has a car above the average quality would withdraw their cars thus lowering the average quality and hence the average price that people are willing to pay falls etc. Here asymmetric information means that one side of the market (buyers) instead of being a diverse group with different willingnesses to pay acts like one homogenous buyer who has a willingness to pay equal to the average buyer. Similarly the diverse group of sellers is treated as if they are a homogenous group of one quality. The issue is that neither side are in fact made up of a homogenous type but their preferences/characteristics are homogenised.
This has similarities to reflexive systems in the sense that there is some variable that people have a preference on and they can also affect that variable through their behaviour. I have separated them out though because of the proximity of behaviour and effect. In reflexive systems, like neighbourhoods, you have a preference for your neighbours and you are the neighbours of those around you. In the used car example you have a preference on the quality but you actions do not directly change the quality of cars but indirectly change this value through the intermediate step of price.
Conservation laws
There can situations where some property of a group, despite looking interesting/surprising, will be realised regardless of the desires/preferences of the participants.
Does the average person in a country make or receive more phone calls, ignoring those made by robots? You may be thinking about your own experiences right now or even those of the people you know. These facts won’t help you answer this question though, since every phone call has one person making it and one person receiving it the total amount of calls made and received must be the same and so the average person makes and receives an identical number of calls. To say that people are drawn by some strange underlying preferences to make and receive the exact same number of phone calls would seem ridiculous. It’s clearly nothing to do with the people but rather the system itself.
The phone call example may seem somewhat inconsequential but not all of these situation are. One university came out with a report stating that minority students were more likely to study with people of a different race to them compared to non minority students. After the phone example it may jump out at you as to what is wrong with this. Minority students are in the minority and so have to have a higher average number of students they study with of a different race. You are left with a headline that seems important at first but literally tells you nothing about individual preferences.
This is not to say that minorities may in fact be more willing to work with students of different races. It is only trying to get across that looking at the quantitative group behaviour of how they actually pair up is not able to reveal anything about this to you.
Schelling in his books notes how “it is astonishing how many hours of committee meeting’s have been spent on proposals to mix men and women in dormitories, or blacks and whites, or freshmen and sophomores, in ways that violated the simple arithmetic principle that no matter how you distribute them, the numbers in all the dormitories have to add up to the numbers that there are.”
An extreme example is this; every family only wants to have a girl and they will keep having children until they get one and then stop having any more children. To simplify things we will say that there are no twins. What happens to the relative frequencies of girls and boys being born? The answer is that they will be equal.[1] At every stage given that there is a 50/50 division between boys and girls there will be the same number born of each. We see the mathematical identity that boys and girls occur with the same probability wipes away any evidence of this particularly strong preference on the individual level.
Another form of conservation law is the acceleration principle. Here how a dependent value (e.g number of recruits) changes is based of the changes in an independent value (e.g desired number of trained officers). I’ve used the terms dependent and independent not because humans can’t decide to change both values but to represent the temporal progression from one to another. The dependent value changes due to changes in the desired independent value.
If the construction industry is replacing 1% of houses and facilitating 2.5% growth in housing each year, what will have to happen to home building to hit the target of having 25% more houses in 5 years? To get to this figure home building will have to grow at 5% a year on top of the 1% replacement rate. This means that home builders go from building roughly 3.5% of the housing stock each year to 6%. This 2.5% increase in the number of houses desired each year for 5 years has lead to a 100% increase in non replacement homes being built.
Say the army drafted people for 2 years, the first 6 months of which was training. Now suppose they wanted to double the number of trained officers in 6 months. How would their training input change? Well there are 1.5 million soldiers currently not in training so you want to get this up to 3 million in 1/2 year. You would need to recruit 2 million people to achieve this. From the outside it would be a training increase a 4x increase in recruits but that would only double your soldiers.
The above examples are a mathematical necessity, you are trying to effectively double the number of new homes in 5 years so you will need to double the number built. The reason these are interesting is because of the acceleration, a 25% increase in one value leads to a 100% increase in another because. While these may seem clear to many, if people can be blindsided by minority students studying on average with more people of different races then the fact a 25% increase in one value can lead to a 100% increase in another will also likely catch some off guard.
Another type of conservation is exhaustive subdivision. Here every option is explicit so you cannot change one value without changing some other value in the process. One example of this is if all deaths are given a cause, in this case if one cause of death declines in significance some other cause will necessarily have to rise. There is no other option. Another example, and I will have to simplify by saying men can only marry women and visa versa, could be looking at the the number of unmarried men vs women. If you see more unmarried men what does it mean? In the setting outlined above it would mean only that there are more men in the country. Every marriage has both sides fully specified as being one woman and one man and so any difference in unmarried people will only come from differences in the total numbers of both sexes.
A less strict conservation principle is when one group is the determinant of the actions of another group. If this is the case then certain characteristics of the subordinate group can be inherited from the superior group. Again ‘group’ is relatively broadly defined but one example of this could be looking at income discrimination (superior group) and housing segregation (subordinate group). If there is income discrimination between sexes or genders then there may be differences in housing patterns that is due to this and not because of preferences for living with those who are like you even though this is what would be observed in the subordinate group. The discrimination in one area has been conserved in another and so looking at the subordinate group may give you less information than you hope about actual housing preferences.
Pooling and partially separating systems
There are three broad categories of equilibrium; a pooling equilibrium is one where players of different types will act the same under different scenarios so behaviour gives you no informational content to discern between types. A partially separating equilibrium is one where some players will act differently under different scenarios so you can partially tell types apart. A fully separating equilibrium is where the behaviour of agents is enough information for you to be able to fully categorise people correctly based on their actions.
As an example suppose there are two insurance contracts and two types of people, healthy and ill. If both types always pick the same contract then the contracts will give you no information on the persons health. If the ill people always pick contract two and the healthy people randomly pick either then you get some information, namely that if they have picked contract one they are healthy. If healthy people only ever pick contract one and ill people only ever pick contract two you have a separating equilibrium where the contract picked gives you all the information about the health of the person.
Few are truly in a separating equilibrium and yet it is often the first things one considers. If one is not cognisant that there are times when the actions of people do not truly reveal significant information you can be lead astray and be left unable to predict how people will act.
Suppose we see that a city is segregated, is this enough for us to tell that people in the city are discriminatory about who they live with? If we can say that someone is discriminatory can we tell who? With this city it could be that lots of different separating mechanisms can lead to the same final result meaning that without more investigation we most likely have a pooling or partially separating equilibrium. What different preference sets could lead to similar segregations?
There could be organised segregation, such as whites not allowing blacks in their neighbourhoods or visa versa. There could be unintentional segregation if one type puts a much higher preference on things like living further out from the city centre. There could be undirected segregation if both types feel more comfortable with their own type and so self select into neighbourhoods with higher numbers like them.
All of these preference sets can lead to a very similar end result. What that means is that the end result of housing separation alone isn’t indicative of any unique set of preferences but rather many different types could pool into the same result.
This is a slightly different category to the previous ones in the sense that here it is less that there is a divergence between group and individual characteristics but rather an unclear backwards mapping from group to individual characteristics meaning group behaviour can be less predictable.
Aggregation effects
There are some properties you can achieve when you aggregate people together that aren’t possible in individuals.
For example groups can hold genuine unresolvable cognitive dissonance in the form of Concordant cycles. Suppose you have three potential people campaigning for some role, can you take the rankings of a group of people and determine the groups ranking of these people? It seems like a simple ask. After all even though some people may be a bit indecisive it would certainly seem absurd to suggest that you or I literally couldn’t rank the candidates in an order. Yet this is precisely what we can see with groups.
In the example above if you take any two candidates one of them is always preferred to the other. However, when you look at the group as a whole no candidate if the overall preferred choice. What seems like a simple situation has turned into a strange political rock, paper, scissors.
Another example of when groups can act differently their ability to effectively de-bias themselves more than we could expect any individual to, which can make the group better at some task then any member it is made up of. We see this with the wisdom of crowds phenomena. The crowd as a whole is able to average the beliefs of all the members. Here the group is able to achieve effectively a neutral outlook on situations that it may be genuinely impossible/very difficult for individuals to do over a broad array of situations.
We have spoken about decomposability earlier and this is another area it is especially important. If we take the voting cycles then the aggregate behaviour in our example can’t be decomposed into the distinct effects to any degree. It is only when all members come together that we get some phenomena. In the wisdom of crowds example there may be some degree of decomposability, some people are likely better forecasters than others for example, but there is clearly some synergistic information the group as a whole is able to use that individuals can’t.
The origins of preferences
As mentioned at the beginning these systems address logically consistent actions.[2] They can lead to results we might not have expected but on seeing the paths we can understand why agents have acted in the ways they have. What I have not discussed is where these preferences come from. Some systems like conservation law systems it seems to matter less where the preferences came from, these are just situations where our human intuition fails to see some mathematical identity. Other systems though, such as the preference enhancing situations, seem to be based much more preferences that we might be interested in understanding the origins of.
Some preferences will likely be evolutionary. The desire to eat or have shelter are likely in built to some degree. Others though may have other origins that are in themselves interesting to consider. One theory, that we will not go into great depth on here, is mimetic desires. This is Girard’s theory that what we learn from each other is what to desire and that we desire objects or characteristics in our attempt to be more like those we admire. As I mentioned these questions are not the scope of this essay but may well be the topic of future ones.
There are many reasons why the world around you may act in surprising ways but hopefully you can see that not all of the ways that lead to this need be surprising. To come back to Scott Alexanders question of why can’t we just change to make things better, I hope that you can see that there are many things stopping us. Individual choice may overpower group goals, we may not even be able to tell why a group is acting the way it is, there may be feedback loops that are incredibly difficult/impossible to break out of.
Charlie Munger said “Some people collect stamps. I collect insanities and absurdities.” and I think that is a useful frame for this topic as well. You will constantly be surprised by how groups of people act so take the time to collect what happened and see if you can figure out why the people are acting they way they are.
[1] If you’re not convinced by my just saying this then hopefully the graphic below will go someway to convince you. I am being a bit overly simplistic even here though. Since girls and boys have different mechanisms as to what you will do if you have one girls will show up slightly more often the more volatile the proportions of girls and boys born each try is. To see this suppose all initial children were girls, everything would stop there and you would have 100% girls. If all initial children were boys then all families would go again and we would expect from then on there to be around 50/50 boys and girls so boys would end up with only a 2/3 share. As the example shows excess boys get dampened in the figures, excess girls don’t. The answer that the numbers are the same is true though if we make the further assumption that each round will have an exact 50/50 split. That may sound like I’m therefore getting to the answer I want by just defining everything so it is so but that is kind of the point of the conservation systems, they have certain properties as a direct result of how they are defined.
[2] In many of these situations we are dealing with what one might call homo economicus, agents who are fully rational and think through their actions. This is obviously not the case for many people, the reason I have operated under this assumption is two fold. The first is that to do otherwise adds vast amounts of complexity and far fewer usable results. The other reason is that I have taken it as a stronger fact that groups exhibit less irrationality than individuals. Groups can still act in strange seemingly irrational ways, aka the point of this essay, but individual irrationality is dampened in a group. You can have one person irrationally angry but having an entire group irrationally angry is much more likely to be due to some logical coordination than each independently being irrationally angry.
Neil Hacker 