Touchscreen projects
2D virtual reality touchscreen apparatus
We use this apparatus to examine how the brain resolves discrepancies between different kinds of sensory information. Subjects make reaching and pointing movements on the lowest surface while looking into the mirror. Images reflected in the mirror from the projection screen (top) make it appear as if the images are in the same plane as the unseen hands.
Example publications
Hsiao A., Lee-Miller T., Block H.J. (2022). Conscious awareness of a visuo-proprioceptive mismatch: Effect on cross-sensory recalibration. Frontiers in Neuroscience—Perception Science
Abstract: The brain estimates hand position using vision and position sense (proprioception). The relationship between visual and proprioceptive estimates is somewhat flexible: visual information about the index finger can be spatially displaced from proprioceptive information, resulting in cross-sensory recalibration of the visual and proprioceptive unimodal position estimates. According to the causal inference framework, recalibration occurs when the unimodal estimates are attributed to a common cause and integrated. If separate causes are perceived, then recalibration should be reduced. Here we assessed visuo-proprioceptive recalibration in response to a gradual visuo-proprioceptive mismatch at the left index fingertip. Experiment 1 asked how frequently a 70 mm mismatch is consciously perceived compared to when no mismatch is present, and whether awareness is linked to reduced visuo-proprioceptive recalibration, consistent with causal inference predictions. However, conscious offset awareness occurred rarely. Experiment 2 tested a larger displacement, 140 mm, and asked participants about their perception more frequently, including at 70 mm. Experiment 3 confirmed that participants were unbiased at estimating distances in the 2D virtual reality display. Results suggest that conscious awareness of the mismatch was indeed linked to reduced cross-sensory recalibration as predicted by the causal inference framework, but this was clear only at higher mismatch magnitudes (70–140 mm). At smaller offsets (up to 70 mm), conscious perception of an offset may not override unconscious belief in a common cause, perhaps because the perceived offset magnitude is in range of participants’ natural sensory biases. These findings highlight the interaction of conscious awareness with multisensory processes in hand perception.
Abstract: The brain estimates hand position using vision and position sense (proprioception). The relationship between visual and proprioceptive estimates is somewhat flexible: visual information about the index finger can be spatially displaced from proprioceptive information, resulting in cross-sensory recalibration of the visual and proprioceptive unimodal position estimates. According to the causal inference framework, recalibration occurs when the unimodal estimates are attributed to a common cause and integrated. If separate causes are perceived, then recalibration should be reduced. Here we assessed visuo-proprioceptive recalibration in response to a gradual visuo-proprioceptive mismatch at the left index fingertip. Experiment 1 asked how frequently a 70 mm mismatch is consciously perceived compared to when no mismatch is present, and whether awareness is linked to reduced visuo-proprioceptive recalibration, consistent with causal inference predictions. However, conscious offset awareness occurred rarely. Experiment 2 tested a larger displacement, 140 mm, and asked participants about their perception more frequently, including at 70 mm. Experiment 3 confirmed that participants were unbiased at estimating distances in the 2D virtual reality display. Results suggest that conscious awareness of the mismatch was indeed linked to reduced cross-sensory recalibration as predicted by the causal inference framework, but this was clear only at higher mismatch magnitudes (70–140 mm). At smaller offsets (up to 70 mm), conscious perception of an offset may not override unconscious belief in a common cause, perhaps because the perceived offset magnitude is in range of participants’ natural sensory biases. These findings highlight the interaction of conscious awareness with multisensory processes in hand perception.
Wali M., Lee-Miller T., Babu R., Block H.J. (2023). Retention of visuo-proprioceptive recalibration in estimating hand position. Scientific Reports 13(1): 6097.
Abstract: The brain estimates hand position using visual and proprioceptive cues, which are combined to give an integrated multisensory estimate. Spatial mismatches between cues elicit recalibration, a compensatory process where each unimodal estimate is shifted closer to the other. It is unclear how well visuo-proprioceptive recalibration is retained after mismatch exposure. Here we asked whether direct vision and/or active movement of the hand can undo visuo-proprioceptive recalibration, and whether recalibration is still evident 24 h later. 75 participants performed two blocks of visual, proprioceptive, and combination trials, with no feedback or direct vision of the hand. In Block 1, a 70 mm visuo-proprioceptive mismatch was gradually imposed, and recalibration assessed. Block 2 tested retention. Between blocks, Groups 1–4 rested or made active movements with their directly visible or unseen hand for several minutes. Group 5 had a 24-h gap between blocks. All five groups recalibrated both vision and proprioception in Block 1, and Groups 1–4 retained most of this recalibration in Block 2. Interestingly, Group 5 showed an offline increase in proprioceptive recalibration, but retained little visual recalibration. Our results suggested that visuo-proprioceptive recalibration is robustly retained in the short-term. In the longer term, contextual factors may affect retention.
Abstract: The brain estimates hand position using visual and proprioceptive cues, which are combined to give an integrated multisensory estimate. Spatial mismatches between cues elicit recalibration, a compensatory process where each unimodal estimate is shifted closer to the other. It is unclear how well visuo-proprioceptive recalibration is retained after mismatch exposure. Here we asked whether direct vision and/or active movement of the hand can undo visuo-proprioceptive recalibration, and whether recalibration is still evident 24 h later. 75 participants performed two blocks of visual, proprioceptive, and combination trials, with no feedback or direct vision of the hand. In Block 1, a 70 mm visuo-proprioceptive mismatch was gradually imposed, and recalibration assessed. Block 2 tested retention. Between blocks, Groups 1–4 rested or made active movements with their directly visible or unseen hand for several minutes. Group 5 had a 24-h gap between blocks. All five groups recalibrated both vision and proprioception in Block 1, and Groups 1–4 retained most of this recalibration in Block 2. Interestingly, Group 5 showed an offline increase in proprioceptive recalibration, but retained little visual recalibration. Our results suggested that visuo-proprioceptive recalibration is robustly retained in the short-term. In the longer term, contextual factors may affect retention.