The role of context in experiments and models of multisensory decision making

Abstract

The availability of signals from multiple senses is often beneficial for perceptual decisions. To study such benefits, models of multisensory decision-making are typically fed with the behavioural performance as measured separately with unisensory component signals. Critically, by doing so, the approach implicitly makes the so-called context invariance assumption, which states that processing of a signal is independent of the experimental context in which it is embedded. However, context invariance is not necessarily true and is difficult to test directly. Here, we aim to assess context invariance indirectly in two testable scenarios. First, to consider the role of stimulus context, we compared unisensory performance in trials that either included a task-irrelevant signal in another modality, or not (unisensory vs. multisensory signal trials). We found that performance was faster but less sensitive in trials that contained a task-irrelevant signal. Hence, stimulus context invariance was violated. Second, to consider the role of instruction context, we compared unisensory performance when participants were asked to detect targets from either one or two modalities (unisensory vs. multisensory instructions). We found that performance was deteriorated in multi- compared to unisensory instructions, which was largely due to modality switch costs in multisensory instructions. Hence, instruction context invariance did not hold either. As performance was variant in both scenarios, context invariance cannot generally assumed to be true. We conclude that models of multisensory decision making should critically consider potential violations of the context invariance assumption as a potentially confounding factor.