My interest in modeling simple dynamics generating complex collective phenomena stem from a root desire to find practical solutions to collective problems such as climate change. For many years, my idea of a practical approach to collective problem solving was helping community organizations create “logic models” of what change they expect, a set of “IF actions A & B THEN we get output C” type of model.  Often the tendency in such logic modeling was to add all kinds of complications, such as resource disparity, seemingly beyond the capacity of any individual person or organization to do anything about. I often found myself torn between suggesting we simplify by focusing on the micro-level individual behavior we could address, as opposed to addressing the structural and collective level factors.  In this book, I offer an alternative approach. I offer models simple enough to play as a game with checkers and coins on a board, representing behaviors and resources, but leading to collective phenomena sophisticated enough to require computer modeling to appreciate all the outcomes.

A breakthrough for me occurred when I discovered a model that elegantly represented many of the tensions we see in social life, including individual versus collective interests, cooperation versus selfishness, and also local versus global perspective. The model is simple enough to demonstrate with checkers on a board. Fill all the spaces on a checkerboard with randomly distributed red and black checkers. Every black cooperates and every red defects. Every checker gets one unit of benefit for every cooperator (black) surrounding them. Every cooperating checker also pays one unit cost per neighbor, which is the cost of cooperating.  Then every checker “imitates” the local neighbor individual with the highest score. So that if a cooperator finds he has the highest score he stays a cooperator, but if he has a defecting neighbor who did better than himself or his seven other neighbors, then he defects—we replace that black checker with a red checker (from a bag with plenty of extra checkers). This selection of the locally best behavior is analogous to evolutionary selection. Repeating these types of rules over and over makes this game “evolutionary.”  One sees what behaviors evolve in the population of individuals inhabiting this checkerboard. We can see how many individuals cooperate and how many defect, and in what spatial patterns. Then repeating the game again and again for different values of benefit and cost and we see how the patterns of behaviors evolve in our board game. We see cooperation thrive in local clusters, communities, even if globally speaking there are more defectors than cooperators. We see different levels of total accumulated wealth (total benefits earned minus costs paid), local and global disparities (clustering of wealth).

I found many ways to incorporate health into these types of evolutionary games, to model various behaviors, including health behaviors, in the context of cooperation, resource distribution, and local community self-organization.  Repeating the game plays over and over reveals emergence of some very interesting collective behaviors, including cooperative health protective action and various types of bottom-up organization. I had found simple models that explained a great many complexities in ecosystems, human behavior, and social ecological dynamics. The same approach works for public health behavior. Social eating, exchange of goods, or any repeated act of taking in resources and paying costs can evolve into complex and unexpected results, when physiological or social feedback is involved. Taking this approach, I found or developed many evolutionary games relevant to various biological, ecological and public health behavioral phenomena. Another game with red and black checkers shows the emergence of collective intolerance for diversity and the emergence of neighborhood segregation (clustering by color) despite individual tolerance.  Some models even apply to the microbial level behavior, for example, a simple rock-paper-scissors game, played with three colors of chips on a board is a good model of the emergence of tolerance for diversity among bacteria.

Seeing parallels and connections between microbial and human behavior was particularly satisfying for me as a way to reconnect to my earliest training.  I had started out aspiring to address public health disparities at the micro-biological level, hoping to find cures for infectious diseases afflicting less fortunate countries.  In those days the following sentence was getting much attention: “Nothing in biology makes sense except in light of evolution” (Dobzhansky 1973, reprinted 1983). But evolutionary processes seemed to me irrelevant to immediate and concrete problems of disease. I was still very far from making the connection to human behavioral ecology. I was lost in the micro-level details of the laboratory. Until, my lab partner challenged me, questioning the relevance of biology to the health of developing countries, “Isn’t that more a political and economic issue?” Hearing that amidst the test tubes and Bunsen burners made the idea more appealing. I turned to community public health.  Yet there was no escaping these issues of individual part versus collective whole, whether in the biology lab or in the field of public health. A parallel tension is between focusing on a local level, a specific community, or a larger federal or state level.  In “international public health,” the debate is between intervening at the level of an individual nation or region or at the global level. These debates and issues also permeate economics, sociology and politics, expressed most generally as a tension between individualism versus collectivism, self-interest competing with collective welfare.  That dilemma is a theme of many game models.  In fact, many features of game models are consistent with a population perspective on health and behavior. In evolutionary games, each individual’s “fitness” depends upon their behavior and also upon the behaviors of others. Ultimately, whether or not behaviors are healthy, sustainable and equitable depends upon the actions of the entire population.

We introduce an ecosystem of rocks, papers, and scissors, each existing in large numbers in a network. Each type is a predator to another, which means they all try to eat one other type. Yet the ecosystem shows remarkable tolerance. Our game Little Tip is a simpler version of the model. This website includes a computer simulation of Little Tip. The chapter focuses instead on an evolutionary model of rock-paper-scissors, and describes how a collective tolerance emerges from individual intolerance.

By contrast, our game Tolerance shows how segregation emerges even when individuals are relatively tolerant. The game includes an alternative version, Attraction, in which integration emerges from gameplay. One lesson is that even a relatively heavy dose of tolerance, a slight preference for one’s own kind aggregates to a collective segregation, what seems a group intolerance. In the alternative game, Attraction, integration can emerge despite that individuals have only a mild attraction to difference.

Tolerance has simple rules, but leads to complex effects, specifically the emergence of segregation. Our computer model of the Tolerance game demonstrates the effect. Our inspiration for the game was Schelling’s model of segregation (1971), so we provide an example of that in a computer model, and discuss this at some length in our rulebook. Also see our blog post on Tolerance. Compare Tolerance to Attraction by playing both games, or play our computer model to compare the two. Use slightly more complicated models to see what happens under varying circumstances.

I have applied the dynamics seen in Meadowkin and Glitterberries and Little Tip to health behavior, showing oscillations in health behavior, and published the results in an article called, “Resilience, tipping, and hydra effects.”

In this chapter, I show how even very simple health behavior dynamics can lead to oscillations which are chaotic. I called this model the Infection-protection game (and created a board game version, soon to be available).

Physiology often exhibits non-linear, fractal patterns of adaptation. I show that such patterns of adaptation also characterize collective health behavior in a model of collective health protection in which individuals use highest payoff biased social learning to decide whether or not to protect against a spreading disease, but benefits of health are shared locally. This model results in collectives of protectors with an exponential distribution of sizes, smaller ones being much more likely. This distribution of protecting collectives, in turn, results in incidence patterns often seen in infectious disease which, although they seem to fluctuate randomly, actually have an underlying order, a fractal time trend pattern. The time trace of infection incidence shows a self-similarity coefficient consistent with a fractal distribution and anti-persistence, reflecting the negative feedback created by health protective behavior responding to disease, when the benefit of health is high enough to stimulate health protection. When the benefit of health is too low to support any health protection, the self-similarity coefficient shows high persistence, reflecting positive feedback resulting the unmitigated spread of disease. Thus the self-similarity coefficient closely corresponds to the level of protection, demonstrating that what might otherwise be regarded as “noise” in incidence actually reflects the fact that protecting collectives form when the spreading disease is present locally but drop protection when disease subsides locally, mitigating disease intermittently. These results hold not only in a deterministic version of the model in a regular lattice network, but also in small-world networks with stochasticity in infection and efficacy of protection. The resulting non-linear and chaotic patterns of behavior and disease cannot be explained by traditional epidemiological methods but a simple agent-based model is sufficient to produce these results.

A public health dilemma results when individuals believe they can personally benefit by not taking health protective action that would benefit others. When individuals face a short-term incentive to avoid the costs associated with protective action they may decide not to protect against a virus. Unfortunately such “defection” can allow a virus to spread, thereby lowering the collective benefit available in the longer term. I model this problem as an evolutionary game, what I call an “infection-protection game.”

Demo_Infection-Protection-Game_Keane_N5_0

Simple logistic with bifurcacion N5

My fullest description of this game model is the below article:

Keane C. Chaos in Collective Health: Fractal Dynamics of Social Learning. Journal of Theoretical Biology 2016), pp. 47-59.

See a movie of the results when starting with one infected-defector or starting with one infected protector.

More fundamental models of chaos include the dynamics of the logistic equation. Experiment with the logistic model or watch a movie of the model output.

Throughout Modeling Behavior, I use evolutionary game models. These models show emergent organization and learning. In chapter 4 of that book, I provide the basis for game modeling of health behavior. In these models individuals learn rules of successful nearest neighbors.

With a simple logic-based grammar of conditional actions, using IF-THEN statements, we can describe shared conditions and interaction. We also can use such logic-based, condition action statements to describe individual and social behavior, and even the intentions and beliefs. We examine how to model intentions, perceived self-efficacy, perceived efficacy of actions, and risk perceptions, and perceived benefits and costs associated with the actions Learn_benefit_&_self-efficacy_expend_N5. The modeling of needs and intentions, and need and condition satisfaction, is one of the most significant yet simple rules we learn, for it connects the sometimes elusive concept of intention to a very concrete process of balancing feedback, otherwise known as negative feedback, that can take action to adjust conditions to meet specified conditions. Adding in perceptions of efficacy modifies what individuals believe they actually can achieve, all in terms of conditional probabilities of achieving benefits. We then use our logic-based language to model the links between behavior, intention and beliefs, and to specify how benefits and costs change when individuals following these rules of behavior interaction. These rules of interaction are very much like the rules we learn when playing a game. First, we have the formal game rules, the rules of what we can do or not, including some of the interactions that are allowed and the consequences. Then we also learn what rules work best in various situations, various types of games in life.

Here’s an example of repeated behavior applied to catching a ball.

What happens when balancing forces interact? What emerges when we have interaction between individuals all striving for some type of balance?

Here’s a simple example:  When an individual feels a need, he takes action to satisfy. If hungry then eat. Such individual behavior can maintain physiological balance. However, suppose the perceived need also spreads socially. An individual’s perceived need increases every time he sees another person satisfying his own need, e.g., when he eats candy. We can represent individuals as squares on a checkerboard and their level of perceived need as a stack of quarters, ranging from no quarters, representing zero perceived need, to four quarters, a perceived need high enough to trigger action. Imagine any single individual starts to build up a perceived need, perhaps because they are susceptible to advertisements for candy. Add quarters one by one to this individual’s square representing the individual’s perceived need for the candy. The candy costs a dollar, and the number of quarters on the stack represents the level of their perceived need, their willingness to pay for the candy. Once the need reaches four quarters worth, the individual buys and consumes the candy, satisfying their hunger and so reducing the perceived need back down to zero (we remove their quarters).  This action (consumption of candy) is visible to the four neighbors, left, right, above and below on the checkerboard.  Every time neighbors see an action, their own perceived need builds up, which we represent by adding another quarter. Once an individual sees the behavior four times, either from the same individual over time or from several neighbors, their need for the candy reaches the threshold necessary to act. We will explore many variants and complications including the metabolic balancing and reward circuitry involved in wanting sugary foods. But we need to understand simple components of the model fully, taking the time to observe any complexity that might emerge.  Do we understand this very simple model well enough to predict what will happen? Our individuals are following two simple conditional action rules, need buildup (IF neighbor takes action THEN add 1 to my perceived need) and need satisfaction (IF my need reaches 4 THEN take action, reducing my perceived need to 0). All we do, as outside experimenters, is stimulate the need of one person, adding quarters one by one to an individual’s patch (or to several randomly distributed individuals). This outside influence represents some stimulation of need other than imitation, perhaps the influence of media. We have all the individuals follow the above two rules, over and over.

This model may seem too simple to be useful, but do we know what will happen? What will the collective behavior be over time? The results turn out to be a good example of a complex system, one generating a great diversity of collective behaviors. We will review this and several variants later in this book, but here I use it to point out the surprising collective complexity emerging from simple rules. The collective behavior starts out looking like a linear increase in the number of persons eating candy, but suddenly we will see an explosion of simultaneous behavior, an avalanche of candy eating, with great numbers of individuals suddenly eating candy all at once. Sandquake-8-neighbors-efficacy-need_N5 We then see behavioral explosions of all sizes, very small and large avalanches of candy eating. From one day to the next day, we see dramatically different distributions of behaviors, a great diversity of numbers of individuals engaging at any given time, and also a great diversity of patterns of social networks.

Here’s a network model, specifically a virus spread on different types of networks (click here).

So far this book has examined many cases of collective learning of health behaviors through conditional-interaction.  We can apply the same conditional interaction modeling approach at the individual level. Just as collectives learn to coordinate their individual actions, so any single individual learns to coordinate his own actions over time. In fact individuals can learn very complex sequences of conditional-interaction rules. This is quite a feat because there are a practically infinite number of possible combinations of conditional-action sequences one might perform, and much of human health behavior involves learning rather complex sequences of conditional-actions.  Moreover, there is a fundamental problem of coordinating immediate intentions with long-term intentions. Many health goals involve achieving long-term outcomes that require long-sequences of actions, even if this means repeating the same actions over a long period. Immediate needs arise, and individuals must cope with these as well. How do individuals learn to pursue long-term goals, given short-term needs that arise? This is the topic of this chapter, which explores my Temporal Interaction Evaluation model, based on some of what we know about neural learning. It is remarkable how evolutionary processes of selection have given rise to a parallel evolutionary processes of learning at the neural level.

TIE network action-values – white uphill

In many types of systems—biological, mechanical and organizational—having a diversity of agents can improve the chance of a response, at least when there is some way to unify the diversity. Each of several agents separately responds to a condition with a certain probability, a conditional-action probability. More specifically each agent separately detects a condition or responds with a certain probability to that condition. Here, agents autonomously implement one or more probabilistic condition-action rules. Agents that detect a condition, then implement an action. The action could be registering a “positive” cancer result, or actually responding to a condition, depending on what we wish to model. For example, if a condition exceeds a certain threshold, such as a certain prevalence of infections, then an agent detects the outbreak condition and signals an alert. In this chapter we model advantages and costs of diversity in increasing successful detection, prediction and response. With the password that comes with the book, you can access more of the interactive models.

Here’s a neural network learning to detect an emergency (click here).

Here’s a model of redundancy and coordination of three different organization responses.

Prior to any planning and evaluation of interventions, public health planners should carry out needs assessments to understand baseline conditions such as community resources and needs. Such needs often have a cyclic existence, repeatedly arising and repeatedly satisfied by resources to reach a dynamic balance. In this situation needs and need satisfaction can be invisible without seeing the dynamic equilibrium.  I describe in this chapter several models of such dynamics, starting with a simple model of metabolic need and need satisfaction, in which an agent uses up energy at a certain rate, and thus needs energy at a certain threshold rate of supply. This physiological balancing mechanism interacts with a reward-circuit balancing mechanism, to stimulate or inhibit eating behavior.

An important case of one balancing process affecting another is sensitization and desensitization of feedback.  Desensitizing feedback, in the sense of delivering more reward per action, such as more pleasure for each unit of food eaten, is likely to lead to more action. Desensitizing, or “blunting” of one’s feedback response to food leads to more eating, not less, and can lead to obesity.  Over-exposure to food can desensitize feedback in this way, and so physiological balancing is one a mechanism linking environmental conditions and overeating. The same type of desensitization occurs, much more strongly, with certain additive substances. Some researchers call this effect “wanting” more and “liking” less (less “pleasure” response).  Intention is high, but the feedback response is low, leading to more perceived need and less perceived satisfaction of that need. Our interactive balancing models are perfectly suited to these mechanisms, as we have clear definitions of intention, need satisfaction, social-environmental and physiological conditions (context).  We clearly define wanting (desire), relative to the gap between intended and actual conditions. We define pleasure, reward experienced as liking, in terms of a clear neuro-physiological conditional-action mechanism.  Most importantly, we have models of the mechanisms of feedback between these elements of condition, action, predicted reward and actual experienced reward. We apply these mechanisms to model over-eating, using mechanisms we learned earlier including reward-circuitry (chapter 7), sensitivity (chapter 8), and metabolism (chapter 9).

We expand this metabolic need model to consumption of shared resources, public health resources, to satisfy need. We then discuss how to model what happens when agents’ actions affect not only their own condition of need, but also affects the condition of other agents.  Example models show the emergence of collective need and collective action to satisfy need.  After specifying health needs and need satisfaction in terms of IF-THEN statements, we implement these in simulations using NetLogo. With these simulations we can show how perceived need and experience stress vary according to relative rates of consumption of and supply of local resources. This dynamic approach may be indispensable when making allocation decisions, e.g., if we were to intervene by allocating resources based on a static measure of unmet need, we might mistakenly cutoff needed resources to agencies that have been successfully meeting their needs. Shortly after a health resource cutback in Baltimore, a syphilis epidemic arose very suddenly, a popular example of a “tipping point” (Gladwell 2000). One interpretation of that suddenness of outbreak was that the resources had met need, establishing equilibrium in syphilis control, such that a relatively small cutback would “tip” the spread of the infection into epidemic levels.  However, if we apply our behavioral avalanche model (chapter 5), we would predict that many different “tipping points” at all scales could occur at any time. That model shows how a steady increment up or down in need satisfaction could at any time produce large or small landslides or explosions in infections.

I offer examples from my own prior research based largely on questions structured in the conditional-action format necessary to understanding dynamics of public health need satisfaction by networks of agencies.  That research, based on surveys of several hundred public health departments, revealed how directors’ beliefs about satisfying local collective need was conditional upon both the need itself and also upon whether or not other agencies were meeting the need. Of course, these sorts of beliefs open up the possibility of the same type of public health cooperative dilemma we examined earlier, this time at the level of networks of public health organizations (chapter 9). Studying networks of such organizations, we often move from complexity to complication. One problem with the word complex is that it evokes complication, but these are distinct concepts.   Complex systems arise when collective properties emerge from simple rules. Complicated systems, by contrast, do not necessarily have at root some simple underlying action. Specific public health or human service agencies have their own unique characteristics, stemming from unique histories in their local jurisdiction. A network of public health organizations, as in any given county, is likely to be quite different from others, despite our many attempts at categorizing types, say local health departments and so on.  Nonetheless, it can be useful to map out networks of organizations, especially as a prelude to developing needs and capacity assessments and program logic models. Here’s a couple relevant models:

Here’s a model of a community resource distribution network.

Here’s a model of emergency response flow networks.

Formally articulated processes of community health needs assessment include mapping “forces for change” (Gilmore and Campbell 2005) and, more generally, stimulating “systems thinking.” We can do this with dynamic modeling of health needs satisfaction. This requires showing the dynamics of satisfying physiological and social-behavioral needs with the needed resources, which in turn requires measurement of local needs, resources and their interaction over time. This interaction over time is the tricky part. Dynamic models sensitize us to the types of dynamic need satisfaction that occur; but, it is difficult to capture change with field techniques of observation and measurement alone (chapter 9).  We can create dynamic models of need including not only individual physiological health (a physiological balancing of metabolic costs and resources) but also, collective health as shared resources and costs. That approach extends naturally to the distribution of resources and costs across populations, as a measure of economic disparity and health disparity.

Program planners and evaluators use logic models to clarify thinking and share ideas about the logical links between inputs and outputs. However, static logic models cannot show many interactions. The diversity of possible combinations and interactions is nearly impossible to show in a printed diagram and is expensive and difficult to study in the field. Interactive logic models, by contrast, can show many different possible combinations of interacting inputs and outputs. Users can interact with the model by choosing different combinations of inputs and observing interactions. Interactions of inputs can result in qualitatively different outputs including synergies and antagonisms, much as in drug interactions. I present several interactive logic models, programmed in a multi-agent modeling environment, which users can access and manipulate through a web-browser. Users can adjust levels of interventions, change the type of organizational interaction, and can change attitudes of the target populations to discover surprising interactions and resulting outcomes, including disparities between subpopulations. Such an approach should lead to improved models, hypotheses, program innovations and training before implementing programs.

Here’s a model of Interventions against Viral Spread – Keane

A simplistic toy model of smoking ban and smoking temptation.

A logic model of a HIV & Hepatitis intervention.

A model of Hepatitis A infection through a restaurant.

Program evaluation (chapter 10) and neural evaluation (chapter 7) balance the same tensions. Both the micro neural level and macro program-level evaluations balance the global with the local, the long-term with the short-term. The performance manager, whether neural or programmatic, faces the need for constant innovation and adaptability while also trying to evaluate performance. This is a tension between flexibly adapting to short-term challenges and long-term intentions, or local with global needs. We discussed these tensions at great length in the prior chapter and proceeded to discuss how interactive logic modeling can help. We also saw how a basic neural mechanism we all have, when healthy, resolves these local-global tensions. It is tempting look to the temporal interaction evaluation model (TIE, chapter 7) for lessons that might aid the program evaluator and performance manager.

One of the most astonishing accomplishments of our TIE neural-computational system is that it automatically coordinates local value with global value, as well as short-term and long-term value, as we discussed in chapter 7. Interestingly, some researchers describe temporal difference type models (similar to TIE) as “online” and even “model-free” to emphasize its remarkable ongoing flexibility in handling the ongoing sudden arising of value opportunities. Yet at the same time, the TIE continuously incorporates some estimate of future value into each immediate action. The neural TIE process has the advantage of constant recursive re-evaluation, similar to the evolutionary, “experimenting society” approach Donald Campbell (1966, 1969, 1973) applied to program evaluation (Campbell and Stanley 1963). When I first studied program evaluation, the standard textbook was Campbell and Stanley’s 1963 class. It was only later that I realized how much more Campbell had published on an evolutionary approach to planning and evaluation. Campbell called himself an “evolutionary epistemologist” (1974), advocating that society-wide program and policy evaluation should follow an evolutionary process of proposing diverse conjectures (“blind variation”) followed by refutation (“selective elimination”) of the experimentally falsified propositions. That is, he advocated selective retention of the variety of intervention innovations we dream up, almost blindly. We throw out the ones we falsify (Campbell 1974, 1975; Campbell and Bickhard 2003).  Campbell realized the parallels and conflicts between evolutionary processes at the biological, social and cultural levels (see “On the Conflicts between Biological and Social Evolution and between Psychology and Moral Tradition” 1974). The parallels to Darwinian evolution through natural selection, “the blind watchmaker,” were explicit. One must balance blind variation with selective elimination for many reasons, but both processes are costly in program evaluation and performance management, so more of one may mean less of the other.  More fundamentally, the trade-off is between decentralized innovation and centrally controlled action and logic model-based planning and evaluation of long-term population health.  Moving into the realm of global health, planners must additionally balance local and global at a yet larger scale, in turn requiring models of how world health is possible (Keane 1998, Robertson 1991). Across all scales of human health behavior, from individual action through organizational behavior up through societal and global levels, we see fundamental dilemmas of coordinating the local and global.  This chapter discusses these sorts of issues and presents an interactive model of how local ethnic markers may evolve in the face of global difference. These local ethnic markers may help individuals choose behaviors that are healthy in the context of their local conditions.

Here’s a model the chapter mentions, on ant foraging.

Here’s a model combining norm spread with segregation.

Here’s a model of  quarantine by student Grace Baek, when she was an undergraduate.

Here’s Yuae Park’s model of smoking-cancer.

Undergraduate Katie Joy’s model of community mental illness.

Undergraduate Katie McNulty’s model of condom use.

KW’s community HIV model.

A student-faculty collaboration:  Health behavior change in advance care planning..