Dopamine Economics

From investing in the stock market to trying the new sushi bar down the street, you make decisions every day that balance risks and rewards. Researchers working at the interface of neuroscience and economics—neuroeconomists, as they’ve dubbed themselves—have been watching brains at work to understand this decision-making process. Two studies involving Caltech neuroeconomists have identified certain regions of the brain that are responsible for interpreting risk as well as reward. These regions are controlled by a neurotransmitter called dopamine, which, among other functions, stimulates the brain’s pleasure centers.

While neuroscientists have been studying reward for decades, very little has been known about the brain’s internal representation of risk. In economics, one financial “model assumes risk and reward are computed separately and then integrated,” says Steven Quartz, associate professor of philosophy. “We looked for biological evidence for this model, such as brain signals that correlated with reward and risk.”

Subjects in Quartz’s study played a simple game while lying inside a functional Magnetic Resonance Imaging (fMRI) machine, like the ones doctors use to diagnose torn muscles. Here the fMRI allowed the neuroscientists to observe changes in blood flow in the brain, pinpointing regions that became active during the game.

Each round of the game consisted of the subjects being shown two cards, one at a time, on a video screen. The deck consisted of 10 cards, numbered 1 through 10. Before seeing either of the cards, the subjects placed a $1 bet on whether the second card would be higher or lower than the first. “It was kind of mean. Since they didn’t have any information, it was a 50-50 gamble on every trial,” says Kerstin Preuschoff (PhD ’07), a former grad student in Quartz’s lab and lead author of the study, which appeared in the August 3, 2006, issue of Neuron. Preuschoff is now a postdoc in the lab of Peter Bossaerts, the Hacker Professor of Economics and Management and professor of finance, and the third author of the study.

The researchers observed what happened after the first card had been seen. “I deliberately used numbered cards, so I knew that they knew what the probabilities of the outcomes were. The idea was for the subjects to experience different probabilities,” says Preuschoff. These probabilities, in turn, led to different levels of expected reward and risk. Say you bet your buck that the second card would be higher, and your first draw proved to be the 1. Your sense of expected reward would be at its highest, as any card you could draw next would be a winner. At the same time, your sense of risk would be zilch. But if the first card drawn was a 5, you would have a 50-50 chance of winning, and thus experience maximum risk.

Activity in a dopamine-controlled region called the ventral striatum proved to mirror the levels of both expected reward and risk. Located deep inside the middle of the brain, below the cerebral cortex, the striatum has been associated with movement control (another of dopamine’s functions; in fact, dopamine therapy is a treatment for the tremors of Parkinson’s disease) and reward-related behaviors for decades. But its involvement in judging risk came as a surprise. “We found two signals in this system—first, an immediate reward signal, and then a delayed risk signal,” says Quartz. The risk signal peaked when the second card was shown. Because the subjects weren’t warned when the second card would appear, the researchers speculated that the risk signal might also serve as an unconscious alert to anticipate the resolution of the bet.

Besides explaining stock-market strategies, the researchers hope future studies may illuminate gambling addiction and bipolar disorder. People with these illnesses may have distorted perceptions of risk or reward, which leads them to choose risky behaviors.

Meanwhile, Colin Camerer, the Axline Professor of Business Economics, has been collaborating with researchers at Baylor College of Medicine and George Mason University to study a different type of reward. These neuroeconomists found that dopaminergic systems not only respond to rewards people experience directly, but also to rewards that people imagine could have been theirs.

To understand the distinction, imagine you are investing in the stock market. Each month, you invest the same small portion of your paycheck and watch the market’s activity. Say the market has skyrocketed for a few months, and you are pondering how much to invest next month. In the past, when you invested a small portion of your paycheck, you got modest rewards. But had you been investing half of your earnings, you would have landed a large windfall. So now you decide to go for it, and put more of your next check into the market.

“The empirical fact is that people will often switch to strategies they never picked before. They couldn’t have learned these strategies by reinforcement” from experienced rewards, says Camerer. In these situations, people use imagined rewards, or rewards that could have been theirs, to guide their decision making. This process, called fictive learning, is similar to the emotion of regret. “Regret is essentially the bodily sensation or name we give to fictive learning when there was a better choice than the one we chose.”

Subjects in this study played a similar stock-market game while the fMRI scanned their brains. The researchers matched activity patterns in their brains with the “fictive error,” which was defined as the difference between the best possible reward and the reward actually experienced.

Camerer and colleagues found that activity in the ventral caudate nucleus mirrored the differences between imagined and experienced rewards. The caudate nucleus is a subdivision of the striatum, the region highlighted in the Quartz study. “Almost everything you would naturally call a reward, or an anticipated reward, seems to activate the striatum,” says Camerer. “It’s quite interesting because it means that simply imagining something rewarding might turn on the reward signal.”

Camerer hopes to expand this research to examine how we learn through observing others’ actions. Imitation may be a socially transmitted form of fictive learning. “If I see you do something and I see it makes you smile or see it makes you vomit, then, even though I didn’t have to do it myself, I may learn something from your actions,” says Camerer.

Although the ability to use imagined rewards has obvious advantages, there could be a dark side. “That same capacity for imagination to activate brain areas as powerfully as actual experiences could lead to paranoia, delusions, and phobias. So, as we come to understand fictive learning better, it may help us to understand these mental states.”

The article describing this work appeared in the May 29 issue of The Proceedings of the National Academy of Sciences. The other authors of the article were Terry Lohrenz and P. Read Montague of Baylor College of Medicine and Kevin McCabe of George Mason University. —MT

Michael M. Torrice is a chemistry grad student who uses amino acids not found in nature to study how signals cross the synapses between nerve cells. He is working with Dennis Dougherty, the Hoag Professor of Chemistry.