Judgment Calls

Making decisions when you are uncertain

November 22, 2019

We all face uncertainty, but few actively look for it like Gülden Ülkümen.

The USC Marshall Associate Professor of Marketing studies uncertainty and how people perceive it. If there’s one thing she is sure of, it is that there is more than one type of uncertainty, and people respond according to the type they face. This position runs contrary to most empirical research on decision making, which tends to position uncertainly as single-dimensional.

Ülkümen has been at the forefront of research on uncertainty since she and her longtime co-author Craig R. Fox of UCLA Anderson School of Management, wrote a chapter for Essays in Judgment and Decision Making (Oslo: Universitetsforlaget, 2011). Their work, “Distinguishing Two Dimensions of Uncertainty,” proposed that people make intuitive distinctions between different dimensions of uncertainty: knowable or epistemic uncertainty, which represents the degree to which an event is seen as, in principle, knowable, and random or aleatory uncertainty, which represents the degree to which an event is seen as inherently stochastic.

“This distinction has important consequences not only for understanding behavior but also for policy making,” Ülkümen said.

In 2014 the team went on to win a $500,000 three-year NSF grant to validate measures of uncertainty and to explore judgments and decisions that are affected by both types. The award has supported the production of three publications in five years, a final manuscript draft, two other manuscripts in preparation and data collection in progress on two other papers.

“It has been very fruitful,” Ülkümen said. “I’m pleased with the progress.”

Making predictions

The researchers’ first paper under the grant, “Two Dimensions of Subjective Uncertainty: Clues from Natural Language,” published with Bertram Malle of Brown University in the Journal of Experimental Psychology in 2016, reveals how people talk about uncertainty. “Language is a way to explore how people think,” said Ülkümen.

In the study, the team gathered data from two years of New York Times articles, along with participants’ opinions about which statements more naturally express different uncertain events.

Our studies suggest that people rely on distinct linguistic expressions to communicate their perception of epistemic and aleatory uncertainty,” Ülkümen said. “They choose a different set of words with epistemic uncertainty than with aleatory.” For example, speakers tend to use confidence statements, (e.g., “I am pretty sure that the capital of California is Sacramento.”) to express epistemic uncertainty. To express aleatory uncertainty, they tend to use likelihood statements (e.g., “I think there is a good chance that I’ll win this hand of blackjack.”).

In addition, Ülkümen said the team showed that given the same event, listeners are influenced by whether speakers use confidence versus likelihood statements. “There is a two-way relationship between language and perception,” she said. “Language reflects response, but it can also influence response.”

This work led Ülkümen, Fox and a team of colleagues to begin “The Epistemic-Aleatory Rating Scale (EARS) in Judgment and Choice,” a lexicon that uses machine learning to detect expressions of uncertainty in various contexts. For example, a marketer might want to know why a consumer is uncertain about making a purchase. “We can tell by their language,” Ülkümen said.

Judgment call

As a college undergraduate, Ülkümen studied naval architecture and ocean engineering, but by the time she graduated, she had her eye on a career in investment banking. This led her to graduate school for an MBA, where Ülkümen, “fell in love with consumer behavior.”

In addition to her MBA, Ülkümen earned a PhD in marketing and was appointed to the USC Marshall Department of Marketing in 2007. But she didn’t leave finance behind. In addition to uncertainty, Ülkümen’s research interests include consumer financial decision making, mental accounting, and framing and presentation of information. “As a researcher, I’m interested in how people think,” she said. “I think it’s important to understand how people believe what they believe.”

Toward this end, Ülkümen and her research team looked at the behaviors that reflect either epistemic or aleatory uncertainty. Their third paper funded by the NSF grant, Judgment Extremity and Accuracy under Epistemic versus Aleatory Uncertainty," published in Management Science in 2017, specifically looks at probability estimates. They found that people make extreme probability judgements for events they view as knowable or epistemic. For events they view aleatory or random, people make judgments based on a probability scale.

While in some cases, judgement extremity is helpful because it prompts better discrimination about probabilities, in other cases, “people can become overconfident,” said Ülkümen. “Judgment extremity does not equal judgment accuracy.”

To help improve judgment accuracy, Ülkümen is looking at ways to influence uncertainty. “We know we can use language to influence how people respond to uncertainty,” she said. “Taking the same message and framing it a certain way can

shift perception. We’re working on tools that can help people make more accurate judgments. The literature has long shown that people are prone to excessive confidence in a wide range of contexts, and that such overconfidence can be both costly and difficult to eliminate.”

Ülkümen also is working on defining more than just two dimensions of uncertainty. For example, beliefs about uncertainty in financial well-being are best thought of along three dimensions: random, or inherently unpredictable and determined by chance events; rigged, or knowable and outside of individuals’ control due to systemic factors such as favoritism and discrimination; and rewarding, or knowable and within individuals’ control due to individual factors such as effort.

“The commonly held wisdom is that support for welfare policies depends on political attitudes,” said Ülkümen. “We show that the right messaging can increase support for social welfare policies across the entire political spectrum. The challenge for lawmakers seeking wealth redistribution is finding language that compatible with people’s beliefs.”

Other areas Ülkümen is looking at include religiosity. “Does religion impact uncertainty?” she asked. “What about emotional consequences? We’re looking at anything that helps us better understand each other.”