Professor Marko Nardini marko.nardini@durham.ac.uk
Professor
Towards Human Sensory Augmentation: A Cognitive Neuroscience Framework for Evaluating Integration of New Signals within Perception, Brain Representations, and Subjective Experience
Nardini, Marko; Scheller, Meike; Ramsay, Melissa; Kristiansen, Olaf; Allen, Chris
Authors
Dr Meike Scheller meike.scheller@durham.ac.uk
Assistant Professor
Melissa Ramsay melissa.burnett@durham.ac.uk
Research Assistant
Olaf Kristiansen olaf.kristiansen@durham.ac.uk
PGR Student Doctor of Philosophy
Dr Chris Allen christopher.p.allen@durham.ac.uk
Post Doctoral Research Associate
Abstract
New wearable devices and technologies provide unprecedented scope to augment or substitute human perceptual abilities. However, the flexibility to reorganize brain processing to use novel sensory signals during early sensitive periods in infancy is much less evident at later ages, making integration of new signals into adults’ perception a significant challenge. We believe that an approach informed by cognitive neuroscience is crucial for maximizing the true potential of new sensory technologies. Here, we present a framework for measuring and evaluating the extent to which new signals are integrated within existing structures of perception and experience. As our testbed, we use laboratory tasks in which healthy volunteers learn new, augmented perceptual-motor skills. We describe a suite of measures of (i) perceptual function (psychophysics), (ii) neural representations (fMRI/decoding), and (iii) subjective experience (qualitative interview/micro-phenomenology) targeted at testing hypotheses about how newly learned signals become integrated within perception and experience. As proof of concept, we provide example data showing how this approach allows us to measure changes in perception, neural processing, and subjective experience. We argue that this framework, in concert with targeted approaches to optimizing training and learning, provides the tools needed to develop and optimize new approaches to human sensory augmentation and substitution.
Citation
Nardini, M., Scheller, M., Ramsay, M., Kristiansen, O., & Allen, C. (2025). Towards Human Sensory Augmentation: A Cognitive Neuroscience Framework for Evaluating Integration of New Signals within Perception, Brain Representations, and Subjective Experience. Augmented Human Research, 10(1), Article 1. https://doi.org/10.1007/s41133-024-00075-7
Journal Article Type | Article |
---|---|
Acceptance Date | Oct 12, 2024 |
Online Publication Date | Oct 28, 2024 |
Publication Date | Dec 1, 2025 |
Deposit Date | Oct 30, 2024 |
Publicly Available Date | Oct 30, 2024 |
Journal | Augmented Human Research |
Print ISSN | 2365-4317 |
Electronic ISSN | 2365-4325 |
Publisher | Springer |
Peer Reviewed | Peer Reviewed |
Volume | 10 |
Issue | 1 |
Article Number | 1 |
DOI | https://doi.org/10.1007/s41133-024-00075-7 |
Keywords | Assistive augmentation, Sensory substitution, Sensory augmentation, Perception, Cognitive neuroscience, Wearable computing, Augmented reality, Human-centered computing → Human computer interaction (HCI); Visualization; Accessibility |
Public URL | https://durham-repository.worktribe.com/output/2993627 |
Files
Published Journal Article
(2.1 Mb)
PDF
Publisher Licence URL
http://creativecommons.org/licenses/by/4.0/
You might also like
Self as a prior: The malleability of Bayesian multisensory integration to social salience
(2023)
Journal Article
Mate Preferences
(2023)
Book Chapter
Downloadable Citations
About Durham Research Online (DRO)
Administrator e-mail: dro.admin@durham.ac.uk
This application uses the following open-source libraries:
SheetJS Community Edition
Apache License Version 2.0 (http://www.apache.org/licenses/)
PDF.js
Apache License Version 2.0 (http://www.apache.org/licenses/)
Font Awesome
SIL OFL 1.1 (http://scripts.sil.org/OFL)
MIT License (http://opensource.org/licenses/mit-license.html)
CC BY 3.0 ( http://creativecommons.org/licenses/by/3.0/)
Powered by Worktribe © 2024
Advanced Search