https://global.oup.com/academic/covers/pop-up/9780195387247

Sensory Cue Integration

Edited by Julia Trommershauser, Konrad Kording, and Michael S. Landy

Computational Neuroscience Series

A broad introduction to the theory of sensory cue integration
Serves as an introduction to the field and as a reference text
Addressed to researchers in multiple allied fields: perceptual psychology, computational neuroscience, cognitive neuroscience and sensory neurophysiology
Covers theoretical issues and empirical results from both behavior and neurophysiology
Addresses cue integration within a single sensory modality (e.g., multiple visual cues to depth) as well as intra-modal cue integration (e.g., auditory and visual cues to spatial location)

Sensory Cue Integration

Edited by Julia Trommershauser, Konrad Kording, and Michael S. Landy

Computational Neuroscience Series

Description

This book provides an introduction into both computational models and experimental paradigms that are concerned with sensory cue integration both within and between sensory modalities. Importantly, across behavioral, electrophysiological and theoretical approaches, Bayesian statistics is emerging as a common language in which cue-combination problems can be expressed. This book focuses on the emerging probabilistic way of thinking about these problems. These approaches derive from the realization that all our sensors are noisy and moreover are often affected by ambiguity. For example, mechanoreceptor outputs are variable and they cannot distinguish if a perceived force is caused by the weight of an object or by force we are producing ourselves. The computational approaches described in this book aim at formalizing the uncertainty of cues. They describe cue combination as the nervous system's attempt to minimize uncertainty in its estimates and to choose successful actions. Some computational approaches described in the chapters of this book are concerned with the application of such statistical ideas to real-world cue-combination problems, such as shape and depth perception. Other parts of the book ask how uncertainty may be represented in the nervous system and used for cue combination.

The broadening scope of probabilistic approaches to cue combination is highlighted in the breadth of topics covered in this book: the chapters summarize and discuss computational approaches and behavioral evidence aimed at understanding the combination of visual, auditory, proprioceptive, and haptic cues. Some chapters address the combination of cues within a single sensory modality while others address the combination across sensory modalities. Neural implementation, behavior, and theory are considered. The unifying aspect of this book is the focus on the uncertainty intrinsic to sensory cues and the underlying question of how the nervous system deals with this uncertainty.

The book is intended as a reference text for graduate students and professionals in perceptual psychology, computational neuroscience, cognitive neuroscience and sensory neurophysiology.

Sensory Cue Integration

Edited by Julia Trommershauser, Konrad Kording, and Michael S. Landy

Computational Neuroscience Series

Preface

I. Theory and Fundamentals

1. Ideal-Observer Models of Cue Integration
Landy, Banks, & Knill
2. Causal Inference in Sensorimotor Learning and Control
Wei & Körding
3. The Role of Generative Knowledge in Object Perception
Battaglia, Kersten, & Schrater
4. Generative Probabilistic Modeling: Understanding Causal Sensorimotor Integration
Vijayakumar, Hospedales, & Haith
5. Modeling Cue Integration in Clutter
Sahani & Whiteley
6. Recruitment of New Visual Cues for Perceptual Appearance
Backus
7. Combining image signals before 3D reconstruction: The Intrinsic Constraint Model of Cue Integration
Domini & Caudek
8. Cue Combination: Beyond So-Called "Optimality"
Rosas & Wichmann

II. Behavioral Studies

9. Priors and Learning in Cue Integration
Seydell, Knill, & Trommershäuser
10. Combining Vision With Audition and Touch, In Adults and In Children
Burr, Binda, & Gori
11. The Statistical Relationship Between Depth, Visual Cues, and Human Perception
Banks, Burge, & Held
12. Multisensory Perception: From Integration to Remapping
Ernst & Di Luca
13. From Integration to Segregation: When and How the Human Nervous System Combines Crossmodal Sensory Signals
Shams & Beierholm
14. Cues and Pseudocues in Texture and Shape Perception
Landy, Ho, Serwe, Trommershäuser, & Maloney
15. Optimality Principles Apply to a Broad Range of Information Integration Problems in Perception and Action
Michel, Brouwer, Jacobs, & Knill

III. Neural Implementation

16. Self-Motion Perception: Multisensory Integration in Extrastriate Visual Cortex
Fetsch, Gu, DeAngelis, & Angelaki
17. Probing Neural Correlates of Cue Integration
Buneo, Apker, & Shi
18. Computational Models of Multisensory Integration in the Cat Superior Colliculus
Rowland, Stein, & Stanford
19. Decoding the Cortical Representation of Depth
Welchman
20. Dynamic Cue Combination in Distributional Population Code Networks
Natarajan & Zemel
21. A Neural Implementation of Optimal Cue Integration
Ma, Beck, & Pouget
22. Contextual Modulation of Visual Receptive Fields: A Bayesian Perspective
Deneve & Lochmann

Sensory Cue Integration

Edited by Julia Trommershauser, Konrad Kording, and Michael S. Landy

Computational Neuroscience Series

Author Information

Julia Trommershäuser spent three years as a postdoctoral researcher at New York University. From 2004-2009, she was a researcher in the Department of Psychology at Giessen University, Germany, funded by an Emmy-Noether Research Award by the German Science Foundation (DFG). She currently works at the Center for Neural Science at New York University on questions in the field of human decision-making under risk and uncertainty, trying to understand how the brain makes use of sensory processing, feedback and prior knowledge.

Konrad Körding is an Assistant Professor at Northwestern University. His lab focuses on the statistical processing of information in the sensorimotor system. He conducts behavioral experiments in humans to ask how people deal with uncertainty and he collaborates with physiologists to ask how the brain does it.

Michael Landy is a Full Professor of Psychology and Neural Science at New York University. His research concerns visual and visuomotor behavior, often comparing human performance to that of a statistically ideal participant. He works in a number of different research areas including spatial visual coding (e.g., texture segmentation), depth perception (including depth cue integration) and movement planning under risk.

Contributors:

Dora Angelaki
Washington University School of Medicine
Anatomy and Neurobiology

Gregory Apker
School of Biological and Health Systems Engineering
Arizona State University

Benjamin T. Backus
Graduate Program in Vision Science
SUNY College of Optometry

Martin S. Banks
School of Optometry
University of California, Berkeley

Peter W. Battaglia
Brain & Cognitive Sciences
Computer Science & Artificial Intelligence Laboratory
MIT

Jeff Beck
Gatsby Computational Neuroscience Unit
University College London

Paola Binda
Istituto di Neuroscienze del CNR

Ulrik Ravnsborg Beierholm
Gatsby Computational Neuroscience Unit
University College London

Anne-Marie Brouwer
TNO Human Factors

Christopher A. Buneo
School of Biological and Health Systems Engineering
Arizona State University

Johannes Burge
Center for Perceptual Systems
Dept. of Psychology
The University of Texas

David Burr
Istituto di Neuroscienze del CNR

Carrado Caudek
Dipartimento di Psicologia
Università degli Studi di Firenze

Gregory C. DeAngelis
Dept. of Brain & Cognitive Sciences
University of Rochester

Sophie Denève
Group for Neural Theory, DEC
Ecole Normale Supérieure

Massimiliano Di Luca
MPI for Biological Cybernetics

Fulvio Domini
Cognitive and Linguistic Sciences
Brown University
Box 1978

Christopher R. Fetsch
Washington University School of Medicine
Anatomy and Neurobiology

Monica Gori
Italian Institute of Technology
University of Genoa

Yong Gu
Washington University School of Medicine
Anatomy and Neurobiology

Adrian Haith
Dept. of Biomedical Engineering
Johns Hopkins University

Robert T. Held
Joint Graduate Group in Bioengineering
University of California, San Francisco and University of California, Berkeley

Yun-Xian Ho
Dept. of Biomedical Informatics
Vanderbilt University Medical Center

Timothy Hospedales
Dept. of Computer Science
Queen Mary University of London

Robert A. Jacobs
Center for Visual Science
Department of Brain & Cognitive Sciences
University of Rochester

Dan Kersten
Dept. of Psychology
University of Minnesota

David C. Knill
Center for Visual Science
University of Rochester

Konrad Körding
Dept. of Physiology, Physical Medicine and Rehabilitation
Rehabilitation Institute of Chicago
Northwestern University

Michael S. Landy
Dept. of Psychology and Center for Neural Science
New York University

Timm Lochmann
Group for Neural Theory, DEC
Ecole Normale Supérieure

Wei Ji Ma
Dept. of Neuroscience
Baylor College of Medicine

Laurence T. Maloney
Dept. of Psychology and Center for Neural Science
New York University

Melchi Michel
Center for Perceptual Systems
Dept. of Psychology
The University of Texas

Rama Natarajan
Center for Neural Science
New York University

Alexandre Pouget
Center for Visual Sciences
Brain and Cognitive Science Dept

Pedro Rosas
Instituto de Ciencias Biomédicas
Facultad de Medicina
Universidad de Chile

Benjamin Rowland
Department of Neurobiology & Anatomy
Wake Forest University School of Medicine

Maneesh Sahani
Gatsby Computational Neuroscience Unit
University College London

Paul R. Schrater
Dept. of Psychology & Computer Science
University of Minnesota

Sascha Serwe
FernUniversität in Hagen
Institut für Psychologie

Anna Seydell
Laboratory of Integrative Neuroscience and Cognition
Dept. of Physiology & Biophysics
Georgetown University Medical Center

Ladan Shams
Department. of Psychology
University of California, Los Angeles

Ying Shi
School of Biological and Health Systems Engineering
Arizona State University

Terrence R. Stanford
Department of Neurobiology & Anatomy
Wake Forest University School of Medicine

Barry E. Stein
Department of Neurobiology & Anatomy
Wake Forest University School of Medicine

Julia Trommershäuser
Center for Neural Science
New York University

Sethu Vijayakumar
1.28 Informatics Forum
School of Informatics
University of Edinburgh

Kunlin Wei
Dept. of Physiology, Physical Medicine and Rehabilitation
Rehabilitation Institute of Chicago
Northwestern University

Andrew Welchman
School of Psychology
University of Birmingham

Louise Whiteley
Gatsby Computational Neuroscience Unit
University College London

Felix Wichmann
Modellierung kognitiver Prozesse
Technische Universität Berlin
and
Bernstein Center for Computational Neuroscience

Richard Zemel
Department of Computer Science
University of Toronto