Visual cues are especially vital for hearing impaired individuals such as cochlear implant (CI) users to understand speech in noise. Functional Near Infrared Spectroscopy (fNIRS) is a light-based imaging technology that is ideally suited for measuring the brain activity of CI users due to its compatibility with both the ferromagnetic and electrical components of these implants. In a preliminary step toward better elucidating the behavioral and neural correlates of audiovisual (AV) speech integration in CI users, we designed a speech-in-noise task and measured the extent to which 24 normal hearing individuals could integrate the audio of spoken monosyllabic words with the corresponding visual signals of a female speaker. In our behavioral task, we found that audiovisual pairings provided average improvements of 103% and 197% over auditory-alone listening conditions in -6 and -9 dB signal-to-noise ratios consisting of multi-talker background noise. In an fNIRS task using similar stimuli, we measured activity during auditory-only listening, visual-only lipreading, and AV listening conditions. We identified cortical activity in all three conditions over regions of middle and superior temporal cortex typically associated with speech processing and audiovisual integration. In addition, three channels active during the lipreading condition showed uncorrected correlations associated with behavioral measures of audiovisual gain as well as with the McGurk effect. Further work focusing primarily on the regions of interest identified in this study could test how AV speech integration may differ for CI users who rely on this mechanism for daily communication.
Pubmed ID: 35821542 RIS Download
Publication data is provided by the National Library of Medicine ® and PubMed ®. Data is retrieved from PubMed ® on a weekly schedule. For terms and conditions see the National Library of Medicine Terms and Conditions.
Multi paradigm numerical computing environment and fourth generation programming language developed by MathWorks. Allows matrix manipulations, plotting of functions and data, implementation of algorithms, creation of user interfaces, and interfacing with programs written in other languages, including C, C++, Java, Fortran and Python. Used to explore and visualize ideas and collaborate across disciplines including signal and image processing, communications, control systems, and computational finance.
View all literature mentionsOpen source software suite for processing and analyzing human brain MRI images. Used for reconstruction of brain cortical surface from structural MRI data, and overlay of functional MRI data onto reconstructed surface. Contains automatic structural imaging stream for processing cross sectional and longitudinal data. Provides anatomical analysis tools, including: representation of cortical surface between white and gray matter, representation of the pial surface, segmentation of white matter from rest of brain, skull stripping, B1 bias field correction, nonlinear registration of cortical surface of individual with stereotaxic atlas, labeling of regions of cortical surface, statistical analysis of group morphometry differences, and labeling of subcortical brain structures.Operating System: Linux, macOS.
View all literature mentionsA Python-based environment of open-source software for mathematics, science, and engineering. The core packages of SciPy include: NumPy, a base N-dimensional array package; SciPy Library, a fundamental library for scientific computing; and IPython, an enhanced interactive console.
View all literature mentions