Search Papers | Poster Sessions | All Posters
Poster A57 in Poster Session A - Tuesday, August 6, 2024, 4:15 – 6:15 pm, Johnson Ice Rink
An information-theoretic perspective on speed-accuracy trade-offs and set size effects
Shuze Liu1 (), Lucy Lai1, Samuel Gershman1, Bilal Bari2; 1Harvard University, 2Massachusetts General Hospital
Policies, the mappings from states to actions, require memory. The amount of memory is dictated by the mutual information between states and actions, or the policy complexity. High-complexity policies preserve state information and generally lead to greater reward compared to low-complexity policies, which discard state information and require less memory. Under our theory, high-complexity policies incur a time cost: they take longer to decode than low-complexity policies. This naturally gives rise to a speed-accuracy trade-off, in which acting quickly necessitates inaccuracy (via low-complexity policies) and acting accurately necessitates acting slowly (via high-complexity policies). Furthermore, the relationship between policy complexity and decoding speed accounts for set size effects: response times grow as a function of set size because larger sets require higher policy complexity. Across two human experiments, we tested these predictions by manipulating intertrial intervals, environmental regularities, and state set sizes. In all cases, we found that humans are sensitive to time costs when modulating policy complexity. Altogether, our theory suggests that policy complexity constraints may underlie some speed-accuracy trade-off and set size effects.
Keywords: Policy compression Speed-accuracy trade-off Reinforcement learning Resource-rationality