Search Papers | Poster Sessions | All Posters
Poster A30 in Poster Session A - Tuesday, August 6, 2024, 4:15 – 6:15 pm, Johnson Ice Rink
Exploring optimal risk-sensitive behavior in the Balloon Analogue Risk Task (BART)
Xin Sui1,2, Peter Dayan1,2, Kevin Lloyd1; 1Max Planck Institute for Biological Cybernetics, Tuebingen, Germany, 2University of Tübingen, Tuebingen, Germany
Attitudes towards risk play a crucial role in everyday decision-making as well as in psychiatric disorders like anxiety. The Balloon Analogue Risk Task (BART) provides a behavioral measure of risk preference that has been widely applied to both clinical and nonclinical populations. However, although most versions of BART involve epistemic as well as aleatoric uncertainty, the implications of this for risk-sensitivity have not been explored. We adopt a prominent theoretical framework for understanding risk, namely conditional value-at-risk (CVaR), to elucidate the effect of risk attitudes on optimal exploration and exploitation in a simple instance of BART. In sequential problems, CVaR comes in two different flavors: pCVaR, which precommits to a level of risk at the very first choice; and nCVaR, which re-applies the same risk level at every step in a nested manner. We show that the structure of stochasticity in the BART is such that pCVaR is more risk-averse than nCVaR in a single trial, for the same nominal risk. We also show that risk preferences and prior expectations interact in risk-sensitive exploration across multiple trials. We hope to provide a normative grounding for a more detailed understanding of behavioral variation in the BART.
Keywords: risk-averse reinforcement learning exploration Bayes adaptive Markov decision process balloon analogue risk task