Language experience, particularly from our native language (L1), shapes our perception of other languages around us. The present study examined how L1 experience moulds the initial processing of foreign (L2) tone during acquisition. In particular, we investigated whether learners were able to rapidly forge new neural memory traces for novel tonal words, which was tracked by recording learners' ERP responses during two word acquisition sessions. We manipulated the degree of L1-L2 familiarity by comparing learners with a nontonal L1 (German) and a tonal L1 (Swedish) and by using tones that were similar (fall) or dissimilar (high, low, rise) to those occurring in Swedish. Our results indicate that a rapid, pre-attentive memory trace build-up for tone manifests in an early ERP component at ~50 ms but only at particularly high levels of L1-L2 similarity. Specifically, early processing was facilitated for an L2 tone that had a familiar pitch shape (fall) and word-level function (inflection). This underlines the importance of these L1 properties for the early processing of L2 tone. In comparison, a later anterior negativity related to the processing of the tones' grammatical content was unaffected by native language experience but was instead influenced by lexicality, pitch prominence, entrenchment, and successful learning. Behaviorally, learning effects emerged for all learners and tone types, regardless of L1-L2 familiarity or pitch prominence. Together, the findings suggest that while L1-based facilitation effects occur, they mainly affect early processing stages and do not necessarily result in more successful L2 acquisition at behavioral level.

Lexical tones are known to be a challenging aspect of speech to acquire in a second language, but several factors are known to affect tone learning facility, such as L1 tonal status (whether a learner's L1 is tonal or not), tone type (the shape of the tones to be acquired), and individual extralinguistic factors (such as musicianship, pitch aptitude, and working memory). Crucially, most of our knowledge of the effect of these factors is based on evidence from perception. The production side of tone learning and the origins of individual variability in learning facility remain relatively understudied. To this end, this study investigated non-native tone production-both in terms of phonetic accuracy in a pseudoword imitation task and in terms of phono-lexical accuracy in a picture-naming task-by English-L1 and Mandarin-L1 speakers. Results show that L1 tonal status and tone type dynamically affected both imitation and picture-naming accuracy, as there were specific accuracy patterns for the English and Mandarin groups. Production accuracy was further facilitated by individual musical experience, working memory, and pitch aptitude. This study's findings add to the currently limited literature on how both language-specific and individual extralinguistic factors modulate non-native tone processing in the speaking modality.

The amplitude of the event-related N1 wave decreases with repeated stimulation. This repetition-attenuation has not previously been investigated in response to variable auditory stimuli, nor has the relative impact of acoustic vs. perceptual category repetition been studied. In the present study, N1 repetition-attenuation was investigated for speech and spectrally rotated speech with varying degrees of acoustic and perceptual category variation. In the speech condition, participants (n = 19) listened to stimulus trains consisting of either the same vowel exemplar (no variability condition), different exemplars of the same vowel (low variability condition), or different exemplars of two different vowels (high variability condition). In the rotated speech condition, the spectrally rotated counterparts of the vowels were presented. Findings show N1 repetition-attenuation in the face of acoustic and perceptual category variability, but no impact of the degree of variability on the degree of N1 attenuation. Speech stimuli resulted in less attenuation than the acoustically matched non-speech stimuli, which is in line with previous findings. It remains unclear if the attenuation of the N1 wave is reduced as a result of stimuli being perceived as belonging to perceptual categories or as a result of some other characteristic of speech.

In the present study, we examine the interactive effect of vowels on Mandarin fricative sibilants using a passive oddball paradigm to determine whether the HEIGHT features of vowels can spread on the surface and influence preceding consonants with unspecified features. The stimuli are two pairs of Mandarin words ([sa] ∼ [ʂa] and [su] ∼ [ʂu]) contrasting in vowel HEIGHT ([LOW] vs. [HIGH]). Each word in the same pair was presented both as standard and deviant, resulting in four conditions (/standard/[deviant]: /sa/[ʂa] ∼ /ʂa/[sa] and /su/[ʂu] ∼ /ʂu/[su]). In line with the Featurally Underspecified Lexicon (FUL) model, asymmetric patterns of processing were found in the [su] ∼ [ʂu] word pair where both the MMN (mismatch negativity) and LDN (late discriminative negativity) components were more negative in /su/[ʂu] (mismatch) than in /ʂu/[su] (no mismatch), suggesting the spreading of the feature [HIGH] from the vowel [u] to [ʂ] on the surface. In the [sa] ∼ [ʂa] pair, however, symmetric negativities (for both MMN and LDN) were observed as there is no conflict between the surface feature [LOW] from [a] to [ʂ] and the underlying specified feature [LOW] of [s]. These results confirm that not all features are fully specified in the mental lexicon: features of vowels can spread on the surface and influence surrounding unspecified segments.

Unlike languages where consonant duration is used contrastively to distinguish word meanings, long consonants in Mandarin Chinese only occur across morpheme boundaries as a result of concatenation and are referred to as fake geminates. To investigate whether Mandarin speakers employ duration contrast to differentiate fake Mandarin geminates and corresponding singletons as well as the underlying pattern of the processing, two auditory oddball tasks were carried out to measure the component of MMN, an index of the automatic detection of deviant stimulus. Mandarin pseudoword pairs which differ only in the duration of the medial consonant ([an1 an1] ∼ [an1 nan1] vs. [an2 an2] ∼ [an2 nan2]) were used as stimuli. An asymmetric pattern of brain activation was observed where the singleton deviant in the context of geminate words elicited higher MMNs than in the reversed condition. These findings are in line with earlier research suggesting that the singleton is unspecified for a moraic representation, while the geminate is specified. Mandarin speakers can employ duration contrast to distinguish fake geminates and corresponding singletons; furthermore, the processing of fake concatenated geminates in contrast to singletons is similar to that of real geminates and corresponding singletons.

The present study investigated the relationship between linguistic tone processing and cortical thickness of bilateral planum temporale (PT) and pars opercularis of the inferior frontal gyrus (IFGpo). Swedish tones on word stems function as cues to upcoming endings. Correlating structural brain imaging data with participants' response time patterns for suffixes, we found that thicker cortex in the left PT was associated with greater reliance on tones to anticipate upcoming inflections on real words. On inflected pseudoword stems, however, the cortical thickness of left IFGpo was associated with tone-suffix processing. Thus cortical thickness of the left PT might play a role in processing tones as part of stored representations for familiar speech segments, most likely when inflected forms are accessed as whole words. In the absence of stored representations, listeners might need to rely on morphosyntactic rules specifying tone-suffix associations, potentially facilitated by greater cortical thickness of left IFGpo.

People learn new languages with varying degrees of success but what are the neuroanatomical correlates of the difference in language-learning aptitude? In this study, we set out to investigate how differences in cortical morphology and white matter microstructure correlate with aptitudes for vocabulary learning, phonetic memory, and grammatical inferencing as measured by the first-language neutral LLAMA test battery. We used ultra-high field (7T) magnetic resonance imaging to estimate the cortical thickness and surface area from sub-millimeter resolved image volumes. Further, diffusion kurtosis imaging was used to map diffusion properties related to the tissue microstructure from known language-related white matter tracts. We found a correlation between cortical surface area in the left posterior-inferior precuneus and vocabulary learning aptitude, possibly indicating a greater predisposition for storing word-figure associations. Moreover, we report negative correlations between scores for phonetic memory and axial kurtosis in left arcuate fasciculus as well as mean kurtosis, axial kurtosis, and radial kurtosis of the left superior longitudinal fasciculus III, which are tracts connecting cortical areas important for phonological working memory.

Occurring at rates up to 6-7 syllables per second, speech perception and understanding involves rapid identification of speech sounds and pre-activation of morphemes and words. Using event-related potentials (ERPs) and functional magnetic resonance imaging (fMRI), we investigated the time-course and neural sources of pre-activation of word endings as participants heard the beginning of unfolding words. ERPs showed a pre-activation negativity (PrAN) for word beginnings (first two segmental phonemes) with few possible completions. PrAN increased gradually as the number of possible completions of word onsets decreased and the lexical frequency of the completions increased. The early brain potential effect for few possible word completions was associated with a blood-oxygen-level-dependent (BOLD) contrast increase in Broca's area (pars opercularis of the left inferior frontal gyrus) and angular gyrus of the left parietal lobe. We suggest early involvement of the left prefrontal cortex in inhibiting irrelevant left parietal activation during lexical selection. The results further our understanding of the importance of Broca's area in rapid online pre-activation of words.

Aptitude for and proficiency in acquiring new languages varies in the human population but their neural bases are largely unknown. We investigated the influence of cortical thickness on language learning predictors measured by the LLAMA tests and a pitch-change discrimination test. The LLAMA tests are first language-independent assessments of language learning aptitude for vocabulary, phonetic working memory, sound-symbol correspondence (not used in this study), and grammatical inferencing. Pitch perception proficiency is known to predict aptitude for learning new phonology. Results show a correlation between scores in a grammatical meaning-inferencing aptitude test and cortical thickness of Broca's area (r(30) = 0.65, p = 0.0202) and other frontal areas (r(30) = 0.66, p = 0.0137). Further, a correlation was found between proficiency in discriminating pitch-change direction and cortical thickness of the right Broca homologue (r(30) = 0.57, p = 0.0006). However, no correlations were found for aptitude for vocabulary learning or phonetic working memory. Results contribute to locating cortical regions important for language-learning aptitude.

It has been suggested that actions can provide a fruitful conceptual context for sound symbolism phenomena, and that tight interaction between manual and articulatory processes might cause that hand actions, in particular, are sound-symbolically associated with specific speech sounds. Experiment 1 investigated whether novel words, built from speech sounds that have been previously linked to precision or power grasp responses, are implicitly associated with perceived actions that present precision manipulation or whole-hand grasp tool-use or the corresponding utilisation pantomimes. In the two-alternative forced-choice task, the participants were more likely to match novel words to tool-use actions and corresponding pantomimes that were sound-symbolically congruent with the words. Experiment 2 showed that the same or even larger sound-action symbolism effect can be observed when the pantomimes present unfamiliar utilisation actions. Based on this we propose that the sound-action symbolism might originate from the same sensorimotor mechanisms that process the meaning of iconic gestural signs. The study presents a novel sound-action phenomenon and supports the view that hand-mouth interaction might manifest itself by associating specific speech sounds with grasp-related utilisations.

Previous studies distinguish between right hemisphere-dominant processing of prosodic/tonal information and left-hemispheric modulation of grammatical information as well as lexical tones. Swedish word accents offer a prime testing ground to better understand this division. Although similar to lexical tones, word accents are determined by words' morphosyntactic structure, which enables listeners to use the tone at the beginning of a word to predict its grammatical ending. We recorded electrophysiological and hemodynamic brain responses to words where stem tones matched or mismatched inflectional suffixes. Tones produced brain potential effects after 136 ms, correlating with subject variability in average BOLD in left primary auditory cortex, superior temporal gyrus, and inferior frontal gyrus. Invalidly cued suffixes activated the left inferior parietal lobe, arguably reflecting increased processing cost of their meaning. Thus, interaction of word accent tones with grammatical morphology yielded a rapid neural response correlating in subject variability with activations in predominantly left-hemispheric brain areas.

In the present study, we investigated if individuals with neurogenic speech sound impairments of three types, Parkinson's dysarthria, apraxia of speech, and aphasic phonological impairment, accommodate their speech to the natural speech rhythm of an auditory model, and if so, whether the effect is more significant after hearing metrically regular sentences as compared to those with an irregular pattern. This question builds on theories of rhythmic entrainment, assuming that sensorimotor predictions of upcoming events allow humans to synchronize their actions with an external rhythm. To investigate entrainment effects, we conducted a sentence completion task relating participants' response latencies to the spoken rhythm of the prime heard immediately before. A further research question was if the perceived rhythm interacts with the rhythm of the participants' own productions, i.e., the trochaic or iambic stress pattern of disyllabic target words. For a control group of healthy speakers, our study revealed evidence for entrainment when trochaic target words were preceded by regularly stressed prime sentences. Persons with Parkinson's dysarthria showed a pattern similar to that of the healthy individuals. For the patient groups with apraxia of speech and with phonological impairment, considerably longer response latencies with differing patterns were observed. Trochaic target words were initiated with significantly shorter latencies, whereas the metrical regularity of prime sentences had no consistent impact on response latencies and did not interact with the stress pattern of the target words to be produced. The absence of an entrainment in these patients may be explained by the more severe difficulties in initiating speech at all. We discuss the results in terms of clinical implications for diagnostics and therapy in neurogenic speech disorders.

Grasping and mouth movements have been proposed to be integrated anatomically, functionally and evolutionarily. In line with this, we have shown that there is a systematic interaction between particular speech units and grip performance. For example, when the task requires pronouncing a speech unit simultaneously with grasp response, the speech units [i] and [t] are associated with relatively rapid and accurate precision grip responses, while [ɑ] and [k] are associated with power grip responses. This study is aimed at complementing the picture about which vowels and consonants are associated with these grasp types. The study validated our view that the high-front vowels and the alveolar consonants are associated with precision grip responses, while low and high-back vowels as well as velar consonants or those whose articulation involves the lowering of the tongue body are associated with power grip responses. This paper also proposes that one reason why small/large concepts are associated with specific speech sounds in the sound-magnitude symbolism is because articulation of these sounds is programmed within the overlapping mechanisms of precision or power grasping.

Although phonological alternation is prevalent in languages, the process of perceiving phonologically alternated sounds is poorly understood, especially at the neurolinguistic level. We examined the process of perceiving Mandarin 3rd tone sandhi (T3 + T3 → T2 + T3) with a mismatch negativity (MMN) experiment. Our design has two independent variables (whether the deviant undergoes tone sandhi; whether the standard and the deviant have matched underlying tone). These two independent variables modulated ERP responses in both the first and the second syllables. Notably, despite the apparent segmental conflict between the standard and the deviant in all conditions, MMN is only observed when neither the standard nor the deviant undergoes tone sandhi, suggesting that discovering the underlying representation of an alternated sound could interfere with the generation of MMN. A tentative model with three hypothesized underlying processing mechanisms is proposed to explain the observed latency and amplitude differences across conditions. The results are also discussed in light of the potential electrophysiological signatures involved in the process of perceiving alternated sounds.

Using computer-vision and image processing techniques, we aim to identify specific visual cues as induced by facial movements made during monosyllabic speech production. The method is named ADFAC: Automatic Detection of Facial Articulatory Cues. Four facial points of interest were detected automatically to represent head, eyebrow and lip movements: nose tip (proxy for head movement), medial point of left eyebrow, and midpoints of the upper and lower lips. The detected points were then automatically tracked in the subsequent video frames. Critical features such as the distance, velocity, and acceleration describing local facial movements with respect to the resting face of each speaker were extracted from the positional profiles of each tracked point. In this work, a variant of random forest is proposed to determine which facial features are significant in classifying speech sound categories. The method takes in both video and audio as input and extracts features from any video with a plain or simple background. The method is implemented in MATLAB and scripts are made available on GitHub for easy access.•Using innovative computer-vision and image processing techniques to automatically detect and track keypoints on the face during speech production in videos, thus allowing more natural articulation than previous sensor-based approaches.•Measuring multi-dimensional and dynamic facial movements by extracting time-related, distance-related and kinematics-related features in speech production.•Adopting the novel random forest classification approach to determine and rank the significance of facial features toward accurate speech sound categorization.

Do people predict different aspects of a predictable word to the same extent? We tested prediction of phonological and gender information by creating phonological and gender mismatches between an article and a predictable noun in Italian. Native Italian speakers read predictive sentence contexts followed by the expected noun (e.g., un incidente: 'accident') or another plausible, but unexpected noun, either beginning with a different phonological class (consonant vs. vowel, e.g., uno scontro: 'collision'; phonological mismatch) or belonging to a different gender class (e.g., un'inondazione: 'flooding'; gender mismatch). Phonological mismatch articles elicited greater negativity than expected articles at posterior channels around 450-800 ms post-stimulus. In contrast, gender mismatch articles elicited greater negativity than expected articles at left posterior channels around 250-800 ms. Unexpected nouns showed an N400 effect followed by frontal positivity relative to expected nouns. The earlier effect for the gender mismatch articles suggests that people are quicker or more likely to pre-activate gender information vs. phonological information of a predictable word. We interpret the results with respect to production-based prediction accounts.

Auditory verbal hallucinations (AVH, 'hearing voices') are an important symptom of schizophrenia but their biological basis is not well understood. One longstanding approach proposes that they are perceptual in nature, specifically that they reflect spontaneous abnormal neuronal activity in the auditory cortex, perhaps with additional 'top down' cognitive influences. Functional imaging studies employing the symptom capture technique-where activity when patients experience AVH is compared to times when they do not-have had mixed findings as to whether the auditory cortex is activated. Here, using a novel variant of the symptom capture technique, we show that the experience of AVH does not induce auditory cortex activation, even while real speech does, something that effectively rules out all theories that propose a perceptual component to AVH. Instead, we find that the experience of AVH activates language regions and/or regions that are engaged during verbal short-term memory.

Understanding speech when background noise is present is a critical everyday task that varies widely among people. A key challenge is to understand why some people struggle with speech-in-noise perception, despite having clinically normal hearing. Here, we developed new figure-ground tests that require participants to extract a coherent tone pattern from a stochastic background of tones. These tests dissociated variability in speech-in-noise perception related to mechanisms for detecting static (same-frequency) patterns and those for tracking patterns that change frequency over time. In addition, elevated hearing thresholds that are widely considered to be 'normal' explained significant variance in speech-in-noise perception, independent of figure-ground perception. Overall, our results demonstrate that successful speech-in-noise perception is related to audiometric thresholds, fundamental grouping of static acoustic patterns, and tracking of acoustic sources that change in frequency. Crucially, speech-in-noise deficits are better assessed by measuring central (grouping) processes alongside audiometric thresholds.

Aberrant pauses are characteristic of schizophrenia and are robustly associated with its negative symptoms. Here, we found that pause behavior was associated with negative symptoms in individuals at clinical high risk (CHR) for psychosis, and with measures of syntactic complexity-phrase length and usage of determiners that introduce clauses-that we previously showed in this same CHR cohort to help comprise a classifier that predicted psychosis. These findings suggest a common impairment in discourse planning and verbal self-monitoring that affects both speech and language, and which is detected in clinical ratings of negative symptoms.

Second language (L2) learners are often aware of the typical pronunciation errors that speakers of their native language make, yet often persist in making these errors themselves. We hypothesised that L2 learners may perceive their own accent as closer to the target language than the accent of other learners, due to frequent exposure to their own productions. This was tested by recording 24 female native speakers of German producing 60 sentences. The same participants later rated these recordings for accentedness. Importantly, the recordings had been altered to sound male so that participants were unaware of their own productions in the to-be-rated samples. We found evidence supporting our hypothesis: participants rated their own altered voice, which they did not recognize as their own, as being closer to a native speaker than that of other learners. This finding suggests that objective feedback may be crucial in fostering L2 acquisition and reduce fossilization of erroneous patterns.