The prelimbic prefrontal cortex (PL) has consistently been found to be necessary for the acquisition of goal-directed actions in rodents. Nevertheless, the specific cellular processes underlying this learning remain unknown. We assessed changes in learning-related expression of mitogen-activated protein kinase/extracellular signal-related kinase (MAPK/ERK1/2) phosphorylation (pERK) in layers 2-3 and 5-6 of the anterior and posterior PL and in the population of neurons projecting to posterior dorsomedial striatum (pDMS), also implicated in goal-directed learning. Rats were given either a single session of training to press a lever for a pellet reward or yoked reward deliveries without instrumental training and assessed 5 or 60 min after training for pERK expression. Relative to yoked training, instrumental training produced an increase in pERK expression in all regions of the PL both at 5 and 60 min, and this was accompanied by an increase in nuclear pERK expression in the posterior PL in rats given instrumental training. pDMS-projecting neurons showed a transient increase in pERK expression in posterior layer 5-6 projection neurons after 5 min, and a delayed increase in anterior layer 2-3 neurons after 60 min, suggesting that ERK expression in the PL is necessary for the consolidation of goal-directed learning. Consistent with this claim, we found that rats trained on two lever press actions for distinct outcomes and then infused with the MEK inhibitor PD98059 into the PL immediately after training failed to acquire specific action-outcome associations, suggesting that persistent pERK signaling in the PL is necessary for goal-directed learning.

The acquisition of new goal-directed actions requires the encoding of action-outcome associations. At a neural level, this encoding has been hypothesized to involve a prefronto-striatal circuit extending between the prelimbic cortex (PL) and the posterior dorsomedial striatum (pDMS); however, no research identifying this pathway with any precision has been reported. We started by mapping the prelimbic input to the dorsal and ventral striatum using a combination of retrograde and anterograde tracing with CLARITY and established that PL-pDMS projections share some overlap with projections to the nucleus accumbens core (NAc) in rats. We then tested whether each of these pathways were functionally required for goal-directed learning; we used a pathway-specific dual-virus chemogenetic approach to selectively silence pDMS-projecting or NAc-projecting PL neurons during instrumental training and tested rats for goal-directed action. We found that silencing PL-pDMS projections abolished goal-directed learning, whereas silencing PL-NAc projections left goal-directed learning intact. Finally, we used a three-virus approach to silence bilateral and contralateral pDMS-projecting PL neurons and again blocked goal-directed learning. These results establish that the acquisition of new goal-directed actions depends on the bilateral PL-pDMS pathway driven by intratelencephalic cortical neurons.

Cholinergic interneurons (CINs) provide the main source of acetylcholine to all striatal regions, and strongly modulate dopaminergic actions through complex regulation of pre- and post-synaptic acetylcholine receptors. Although striatal CINs have a well-defined electrophysiological profile, their biochemical properties are poorly understood, likely due to their low proportion within the striatum (2-3%). We report a strong and sustained phosphorylation of ribosomal protein S6 on its serine 240 and 244 residues (p-Ser²⁴⁰⁻²⁴⁴-S6rp), a protein integrant of the ribosomal machinery related to the mammalian target of the rapamycin complex 1 (mTORC1) pathway, which we found to be principally expressed in striatal CINs in basal conditions. We explored the functional relevance of this cellular event by pharmacologically inducing various sustained physiological activity states in CINs and assessing the effect on the levels of S6rp phosphorylation. Cell-attached electrophysiological recordings from CINs in a striatal slice preparation showed an inhibitory effect of tetrodotoxin (TTX) on action potential firing paralleled by a decrease in the p-Ser²⁴⁰⁻²⁴⁴-S6rp signal as detected by immunofluorescence after prolonged incubation. On the other hand, elevation in extracellular potassium concentration and the addition of apamin generated an increased firing rate and a burst-firing activity in CINs, respectively, and both stimulatory conditions significantly increased Ser²⁴⁰⁻²⁴⁴-S6rp phosphorylation above basal levels when incubated for one hour. Apamin generated a particularly large increase in phosphorylation that was sensitive to rapamycin. Taken together, our results demonstrate for the first time a link between the state of neuronal activity and a biochemical signaling event in striatal CINs, and suggest that immunofluorescence can be used to estimate the cellular activity of CINs under different pharmacological and/or behavioral conditions.

In mammalian species, the capacity for goal-directed action relies on a process of cognitive-emotional integration, which melds the causal and incentive learning processes that link action-goal associations with the current value of the goal [1]. Recent evidence suggests that such integration depends on a cortical-limbic-striatal circuit centered on the posterior dorsomedial striatum (pDMS) [2]. Learning-related plasticity has been described at both classes of principal neuron in the pDMS, the direct (dSPNs) and indirect (iSPNs) pathway spiny projection neurons [3-5], and is thought to depend on inputs from prelimbic cortex (PL) [6] and the basolateral amygdala (BLA) [7]. Nevertheless, the relative contribution of these structures to the cellular changes associated with goal-directed learning has not been assessed, nor is it known whether any plasticity specific to the PL and BLA inputs to the pDMS is localized to dSPNs, iSPNs, or both cell types. Here, by combining instrumental conditioning with circuit-specific manipulations and ex vivo optogenetics in mice, we discovered that the PL and not the BLA input to pDMS is pivotal for goal-directed learning and that plasticity in the PL-pDMS pathway was bilateral and specific to dSPNs in the pDMS. Subsequent experiments revealed the BLA is critically but indirectly involved in striatal plasticity via its input to the PL; inactivation of the BLA projection to PL blocked goal-directed learning and prevented learning-related plasticity at dSPNs in pDMS.

Evidence suggests that, in Pavlovian conditioning, associations form between conditioned stimuli and multiple components of the unconditioned stimulus (US). It is common, for example, to regard USs as composed of sensory and affective components, the latter being either appetitive (e.g., food or water) or aversive (e.g., shock or illness) and, therefore, to suppose different USs of the same affective class activate a common affective system. Furthermore, evidence is growing for the suggestion that, in competitive learning situations, competition between predictive stimuli is primarily for association with the affective system activated by the US. Thus, a conditioned stimulus (CS) previously paired with one US will block conditioning to another CS when both are presented together and paired with a different US of the same affective class, a phenomenon called transreinforcer blocking. Importantly, similar effects have been reported when steps are taken to turn the pretrained CS into a conditioned inhibitor, which activates the opposing affective state to the excitor that it inhibits. Thus, an appetitive inhibitor can block conditioning to a second CS when they are presented together and paired with foot shock. Here we show that the same is true of an aversive inhibitor. In two experiments conducted in rats, we found evidence that an aversive inhibitor blocked conditioning to a second CS when presented in a compound and paired with food. Such findings demonstrate that affective processes and their opponency organize appetitive-aversive interactions and establish the valences on which they are based, consistent with incentive theories of Pavlovian conditioning.

In four experiments we assessed the effect of systemic amphetamine on the ability of a stimulus paired with reward and a stimulus that was not paired with reward to support instrumental conditioning; i.e., we trained rats to press two levers, one followed by a stimulus that had been trained in a predictive relationship with a food outcome and the other by a stimulus unpaired with that reward. Here we show, in general accord with predictions from the dopamine re-selection hypothesis [Redgrave and Gurney (2006). Nat. Rev. Neurosci. 7, 967-975], that systemic amphetamine greatly enhanced the performance of lever press responses that delivered a visual stimulus whether that stimulus had been paired with reward or not. In contrast, amphetamine had no effect on the performance of responses on an inactive lever that had no stimulus consequences. These results support the notion that dopaminergic activity serves to mark or tag actions associated with stimulus change for subsequent selection (or re-selection) and stand against the more specific suggestion that dopaminergic activity is solely related to the prediction of reward.

In two experiments we investigated the effects of elevated dopaminergic tone on instrumental learning and performance using dopamine transporter knockdown (DAT KD) mice. In Experiment 1, we showed that both DAT KD mice and wild-type controls were similarly sensitive to outcome devaluation induced by sensory specific satiety, indicating normal action-outcome learning in both groups. In Experiment 2, we used a Pavlovian-to-instrumental transfer procedure to assess the potentiation of instrumental responding by Pavlovian conditional stimuli (CS). Although during the Pavlovian training phase the DAT KD mice entered the food magazine more frequently in the absence of the CS, when tested later both groups showed outcome-selective PIT. These results suggest that the elevated dopaminergic tone reduced the selectivity of stimulus control over conditioned behavior, but did not affect instrumental learning.

Stress reduces cognitive flexibility and dopamine D1 receptor-related activity in the prelimbic cortex (PL), effects hypothesized to depend on reduced corticotropic releasing factor receptor type 1 (CRFr1) regulation of dopamine neurons in the ventral tegmental area (VTA). We assessed this hypothesis in rats by examining the effect of chronic unpredictable restraint stress (CUS), mild acute stress, or their combination on cognitive flexibility, CRFr1 expression in the VTA and D1-related activity in PL. In Experiment 1, rats received either CUS or equivalent handling for 14 days before being trained to press two levers to earn distinct food outcomes. Initial learning was assessed using an outcome devaluation test after which cognitive flexibility was assessed by reversing the outcomes earned by the actions. Prior to each reversal training session, half the CUS and controls receiving acute stress with action-outcome updating assessed using a second devaluation test and CRFr1 expression in the VTA assessed using in-situ hybridisation. Although CUS did not itself affect action-outcome learning, its combination with acute stress blocked reversal learning and decreased VTA CRFr1 expression after acute shock. The relationship between these latter two effects was assessed in Experiment 2 by pharmacologically disconnecting the VTA and PL, unilaterally blocking neurons expressing CRFr1 in the VTA and D1 receptors in the contralateral PL during reversal learning after acute stress. Acute stress again blocked reversal learning but only in the group with VTA-PL disconnection, demonstrating that VTA CRFr1-induced facilitation of dopaminergic activity in the PL is necessary for maintaining cognitive flexibility after acute stress. [250].

Since Cajal's visualisations of the synaptic spine, this feature of the neuron has been of interest to neuroscientists and has been investigated usually in reference to degeneration or proliferation of dendrites and their neurons. Synaptic spine measurement often forms a critical element of any study investigating neuronal morphology. However, the way researchers have counted spines hasn't changed for almost a century. Some of the currently used legacy methods fail to accommodate obscured spines or factor-in visibility differences between histological stains.

Changes in dopaminergic neural function can be induced by an acute inflammatory state that, by altering the integrity of the neurovasculature, induces neuronal stress, cell death and causes functional deficits. Effectively blocking these effects of inflammation could, therefore, reduce both neuronal and functional decline. To test this hypothesis, we inhibited vascular adhesion protein 1 (VAP-1), a membrane-bound protein expressed on the endothelial cell surface, that mediates leukocyte extravasation and induces oxidative stress.

The present experiments examined whether extinction of a stimulus predicting food affects the ability of that stimulus to energize instrumental performance to obtain food. We first used a general Pavlovian instrumental transfer (PIT) paradigm in which rats were first given Pavlovian conditioning with a stimulus predicting one type of food outcome and were then trained to lever press for a different food outcome. We found that the Pavlovian stimulus enhanced performance of the lever press response and that this enhancement was preserved after extinction of that stimulus (Experiment 1) even when the context was manipulated to favor the expression of extinction (Experiment 2). Next, we assessed whether extinction influenced the excitatory effect of a stimulus when it was trained as a discriminative stimulus. Extinction of this stimulus alone had no effect on its ability to control instrumental performance; however, when extinguished with its associated lever press response, discriminative control was lost (Experiments 3 and 4). Finally, after instrumental and Pavlovian training, we extinguished a Pavlovian stimulus predicting one food outcome with a lever press response that delivered a different outcome. In a general PIT test, we found this extinction abolished the ability of the Pavlovian stimulus to elevate responding on a lever trained with a different outcome, revealing for the first time that extinction can abolish the general PIT effect. We conclude that extinction can produce an inhibitory association between the stimulus and the general response type, whether Pavlovian or instrumental, performed during the extinction training.

Bidirectionally aberrant medial orbitofrontal cortical (mOFC) activity has been consistently linked with compulsive disorders and related behaviors. Although rodent studies have established a causal link between mOFC excitation and compulsive-like actions, no such link has been made with mOFC inhibition. Here, we use excitotoxic lesions of mOFC to investigate its role in sensitivity to punishment; a core characteristic of many compulsive disorders. In our first experiment, we demonstrated that mOFC lesions prevented rats from learning to avoid a lever that was punished with a stimulus that coterminated with footshock. Our second experiment demonstrated that retrieval of punishment learning is also somewhat mOFC-dependent, as lesions prevented the extended retrieval of punishment contingencies relative to shams. In contrast, mOFC lesions did not prevent rats from reacquiring the ability to avoid a punished lever when it was learned prior to lesions being administered. In both experiments, Pavlovian fear conditioning to the stimulus was intact for all animals. Together, these results reveal that the mOFC regulates punishment learning and retrieval in a manner that is separate from any role in Pavlovian fear conditioning. These results imply that aberrant mOFC activity may contribute to the punishment insensitivity that is observed across multiple compulsive disorders.

Cognitive impairments contribute significantly to disease burden in young individuals presenting with major psychiatric disorders. The capacity to encode the consequences of one's actions may be of particular importance for real-world functioning due to its fundamental role in goal-directed behavior.

Neuroinflammation is initiated by a variety of stimuli including infections, sepsis, neurodegenerative diseases or traumatic brain injury and, if not adequately controlled, can lead to various degrees of neuronal damage and behavioural impairment. A critical event in the initial steps of inflammation is neutrophil extravasation. Semicarbazide-sensitive amine oxidase (SSAO, also known as vascular adhesion protein 1 or VAP-1) regulates neutrophil adhesion and extravasation. Here, we elucidate the role of SSAO/VAP-1 in the early stage inflammatory response after LPS insult in the brain.

Behavioral evidence suggests that instrumental conditioning is governed by two forms of action control: a goal-directed and a habit learning process. Model-based reinforcement learning (RL) has been argued to underlie the goal-directed process; however, the way in which it interacts with habits and the structure of the habitual process has remained unclear. According to a flat architecture, the habitual process corresponds to model-free RL, and its interaction with the goal-directed process is coordinated by an external arbitration mechanism. Alternatively, the interaction between these systems has recently been argued to be hierarchical, such that the formation of action sequences underlies habit learning and a goal-directed process selects between goal-directed actions and habitual sequences of actions to reach the goal. Here we used a two-stage decision-making task to test predictions from these accounts. The hierarchical account predicts that, because they are tied to each other as an action sequence, selecting a habitual action in the first stage will be followed by a habitual action in the second stage, whereas the flat account predicts that the statuses of the first and second stage actions are independent of each other. We found, based on subjects' choices and reaction times, that human subjects combined single actions to build action sequences and that the formation of such action sequences was sufficient to explain habitual actions. Furthermore, based on Bayesian model comparison, a family of hierarchical RL models, assuming a hierarchical interaction between habit and goal-directed processes, provided a better fit of the subjects' behavior than a family of flat models. Although these findings do not rule out all possible model-free accounts of instrumental conditioning, they do show such accounts are not necessary to explain habitual actions and provide a new basis for understanding how goal-directed and habitual action control interact.

We assessed whether the presence of contextual cues paired with alcohol would disrupt rats' capacity to express appropriate goal-directed action control. Rats were first given differential context conditioning such that one set of contextual cues was paired with the injection of ethanol and a second, distinctive set of cues was paired with the injection of saline. All rats were then trained in a third, neutral context to press one lever for grain pellets and another lever for sucrose pellets. They were then given two extinction tests to evaluate their ability to choose between the two actions in response to the devaluation of one of the two food outcomes with one test conducted in the alcohol-paired context and the other conducted in the control (saline-paired) context. In the control context, rats exhibited goal-directed action control; i.e., they were able selectively to withhold the action that previously earned the now devalued outcome. However, these same rats were impaired when tested in the alcohol-paired context, performing both actions at the same rate regardless of the current value of their respective outcomes. Subsequent testing revealed that the rats were capable of overcoming this impairment if they were giving response-contingent feedback about the current value of the food outcomes. These results provide a clear demonstration of the disruptive influence that alcohol-paired cues can exert on decision-making in general and goal-directed action selection and choice in particular.

Social anxiety disorder is characterized by excessive fear and habitual avoidance of social situations. Decision-making models suggest that patients with anxiety disorders may fail to exhibit goal-directed control over actions. We therefore investigated whether such biases may also be associated with social anxiety and to examine the relationship between such behavior with outcomes from cognitive-behavioral therapy. Patients diagnosed with social anxiety and controls completed an instrumental learning task in which two actions were performed to earn food outcomes. After outcome devaluation, where one outcome was consumed to satiety, participants were re-tested in extinction. Results indicated that, as expected, controls were goal-directed, selectively reducing responding on the action that previously delivered the devalued outcome. Patients with social anxiety, however, exhibited no difference in responding on either action. This loss of a devaluation effect was associated with greater symptom severity and poorer response to therapy. These findings indicate that variations in goal-directed control in social anxiety may represent both a behavioral endophenotype and may be used to predict individuals who will respond to learning-based therapies.

The mammalian neuropeptide oxytocin has well-characterized effects in facilitating prosocial and affiliative behavior. Additionally, oxytocin decreases physiological and behavioral responses to social stress. In the present study we investigated the effects of oxytocin on cognitive appraisals after a naturalistic social stress task in healthy male students. In a randomized, double-blind, placebo-controlled trial, 48 participants self-administered either an oxytocin or placebo nasal spray and, following a wait period, completed an impromptu speech task. Eye gaze to a pre-recorded video of an audience displayed during the task was simultaneously collected. After the speech, participants completed questionnaires assessing negative cognitive beliefs about speech performance. Whilst there was no overall effect of oxytocin compared to placebo on either eye gaze or questionnaire measures, there were significant positive correlations between trait levels of anxiety and negative self-appraisals following the speech. Exploratory analyses revealed that whilst higher trait anxiety was associated with increasingly poorer perceptions of speech performance in the placebo group, this relationship was not found in participants administered oxytocin. These results provide preliminary evidence to suggest that oxytocin may reduce negative cognitive self-appraisals in high trait anxious males. It adds to a growing body of evidence that oxytocin seems to attenuate negative cognitive responses to stress in anxious individuals.

Habits are characterized by an insensitivity to their consequences and, as such, can be distinguished from goal-directed actions. The neural basis of the development of demonstrably outcome-insensitive habitual actions in humans has not been previously characterized. In this experiment, we show that extensive training on a free-operant task reduces the sensitivity of participants' behavior to a reduction in outcome value. Analysis of functional magnetic resonance imaging data acquired during training revealed a significant increase in task-related cue sensitivity in a right posterior putamen-globus pallidus region as training progressed. These results provide evidence for a shift from goal-directed to habit-based control of instrumental actions in humans, and suggest that cue-driven activation in a specific region of dorsolateral posterior putamen may contribute to the habitual control of behavior in humans.

Pathological forms of the microtubule-associated protein tau are involved in a large group of neurodegenerative diseases named tauopathies, including frontotemporal lobar degeneration (FTLD-tau). K369I mutant tau transgenic mice (K3 mice) recapitulate neural and behavioural symptoms of FTLD, including tau aggregates in the cortex, alterations to nigrostriatum, memory deficits and parkinsonism. The aim of this study was to further characterise the K3 mouse model by examining functional alterations to the striatum. Whole-cell patch-clamp electrophysiology was used to investigate the properties of striatal neurons in K3 mice and wildtype controls. Additionally, striatal-based instrumental learning tasks were conducted to assess goal-directed versus habitual behaviours (i.e., by examining sensitivity to outcome devaluation and progressive ratios). The K3 model demonstrated significant alterations in the discharge properties of striatal neurons relative to wildtype mice, which manifested as a shift in neuronal output towards a burst firing state. K3 mice acquired goal-directed responding faster than control mice and were goal-directed at test unlike wildtype mice, which is likely to indicate reduced capacity to develop habitual behaviour. The observed pattern of behaviour in K3 mice is suggestive of deficits in dorsal lateral striatal function and this was supported by our electrophysiological findings. Thus, both the electrophysiological and behavioural alterations indicate that K3 mice have early deficits in striatal function. This finding adds to the growing literature which indicate that the striatum is impacted in tau-related neuropathies such as FTLD, and further suggests that the K3 model is a unique mouse model for investigating FTLD especially with striatal involvement.