Interval schedules of reinforcement are known to generate habitual behavior, the performance of which is less sensitive to revaluation of the earned reward and to alterations in the action-outcome contingency. Here we report results from experiments using different types of interval schedules of reinforcement in mice to assess the effect of uncertainty, in the time of reward availability, on habit formation. After limited training, lever pressing under fixed interval (FI, low interval uncertainty) or random interval schedules (RI, higher interval uncertainty) was sensitive to devaluation, but with more extended training, performance of animals trained under RI schedules became more habitual, i.e. no longer sensitive to devaluation, whereas performance of those trained under FI schedules remained goal-directed. When the press-reward contingency was reversed by omitting reward after pressing but presenting reward in the absence of pressing, lever pressing in mice previously trained under FI decreased more rapidly than that of mice trained under RI schedules. Further analysis revealed that action-reward contiguity is significantly reduced in lever pressing under RI schedules, whereas action-reward correlation is similar for the different schedules. Thus the extent of goal-directedness could vary as a function of uncertainty about the time of reward availability. We hypothesize that the reduced action-reward contiguity found in behavior generated under high uncertainty is responsible for habit formation.

In recent years optogenetics has rapidly become an essential technique in neuroscience. Its temporal and spatial specificity, combined with efficacy in manipulating neuronal activity, are especially useful in studying the behavior of awake behaving animals. Conventional optogenetics, however, requires the use of lasers and optic fibers, which can place considerable restrictions on behavior. Here we combined a wirelessly controlled interface and small implantable light-emitting diode (LED) that allows flexible and precise placement of light source to illuminate any brain area. We tested this wireless LED system in vivo, in transgenic mice expressing channelrhodopsin-2 in striatonigral neurons expressing D1-like dopamine receptors. In all mice tested, we were able to elicit movements reliably. The frequency of twitches induced by high power stimulation is proportional to the frequency of stimulation. At lower power, contraversive turning was observed. Moreover, the implanted LED remains effective over 50 days after surgery, demonstrating the long-term stability of the light source. Our results show that the wireless LED system can be used to manipulate neural activity chronically in behaving mice without impeding natural movements.

Risk is a ubiquitous feature of the environment for most organisms, who must often choose between a small and certain reward and a larger but less certain reward. To study choice behavior under risk in a genetically well characterized species, we trained mice (C57BL/6) on a discrete trial, concurrent-choice task in which they must choose between two levers. Pressing one lever (safe choice) is always followed by a small reward. Pressing the other lever (risky choice) is followed by a larger reward, but only on some of the trials. The overall payoff is the same on both levers. When mice were not food deprived, they were indifferent to risk, choosing both levers with equal probability regardless of the level of risk. In contrast, following food or water deprivation, mice earning 10% sucrose solution were risk-averse, though the addition of alcohol to the sucrose solution dose-dependently reduced risk aversion, even before the mice became intoxicated. Our results falsify the budget rule in optimal foraging theory often used to explain behavior under risk. Instead, they suggest that the overall demand or desired amount for a particular reward determines risk preference. Changes in motivational state or reward identity affect risk preference by changing demand. Any manipulation that increases the demand for a reward also increases risk aversion, by selectively increasing the frequency of safe choices without affecting frequency of risky choices.

We previously reported a new line of Shank3 mutant mice which led to a complete loss of Shank3 by deleting exons 4-22 (Δe4-22) globally. Δe4-22 mice display robust ASD-like behaviors including impaired social interaction and communication, increased stereotypical behavior and excessive grooming, and a profound deficit in instrumental learning. However, the anatomical and neural circuitry underlying these behaviors are unknown. We generated mice with Shank3 selectively deleted in forebrain, striatum, and striatal D1 and D2 cells. These mice were used to interrogate the circuit/brain-region and cell-type specific role of Shank3 in the expression of autism-related behaviors. Whole-cell patch recording and biochemical analyses were used to study the synaptic function and molecular changes in specific brain regions. We found perseverative exploratory behaviors in mice with deletion of Shank3 in striatal inhibitory neurons. Conversely, self-grooming induced lesions were observed in mice with deletion of Shank3 in excitatory neurons of forebrain. However, social, communicative, and instrumental learning behaviors were largely unaffected in these mice, unlike what is seen in global Δe4-22 mice. We discovered unique patterns of change for the biochemical and electrophysiological findings in respective brain regions that reflect the complex nature of transcriptional regulation of Shank3. Reductions in Homer1b/c and membrane hyper-excitability were observed in striatal loss of Shank3. By comparison, Shank3 deletion in hippocampal neurons resulted in increased NMDAR-currents and GluN2B-containing NMDARs. These results together suggest that Shank3 may differentially regulate neural circuits that control behavior. Our study supports a dissociation of Shank3 functions in cortical and striatal neurons in ASD-related behaviors, and it illustrates the complexity of neural circuit mechanisms underlying these behaviors.

The ventral tegmental area (VTA) is a major source of dopamine, especially to the limbic brain regions. Despite decades of research, the function of VTA dopamine neurons remains controversial. Here, using a novel head-fixed behavioral system with five orthogonal force sensors, we show for the first time that the activity of dopamine neurons precisely represents the impulse vector (force exerted over time) generated by the animal. Distinct populations of VTA dopamine neurons contribute to components of the impulse vector in different directions. Optogenetic excitation of these neurons shows a linear relationship between signal injected and impulse generated. Optogenetic inhibition paused force generation or produced force in the backward direction. At the same time, these neurons also regulate the initiation and execution of anticipatory licking. Our results indicate that VTA dopamine controls the magnitude, direction, and duration of force used to move toward or away from any motivationally relevant stimuli.

Giant ankyrin-B (ankB) is a neurospecific alternatively spliced variant of ANK2, a high-confidence autism spectrum disorder (ASD) gene. We report that a mouse model for human ASD mutation of giant ankB exhibits increased axonal branching in cultured neurons with ectopic CNS axon connectivity, as well as with a transient increase in excitatory synapses during postnatal development. We elucidate a mechanism normally limiting axon branching, whereby giant ankB localizes to periodic axonal plasma membrane domains through L1 cell-adhesion molecule protein, where it couples microtubules to the plasma membrane and prevents microtubule entry into nascent axon branches. Giant ankB mutation or deficiency results in a dominantly inherited impairment in selected communicative and social behaviors combined with superior executive function. Thus, gain of axon branching due to giant ankB-deficiency/mutation is a candidate cellular mechanism to explain aberrant structural connectivity and penetrant behavioral consequences in mice as well as humans bearing ASD-related ANK2 mutations.

Terrestrial locomotion presents tremendous computational challenges on account of the enormous degrees of freedom in legged animals, and complex, unpredictable properties of natural environments, including the body and its effectors, yet the nervous system can achieve locomotion with ease. Here we introduce a quadrupedal robot that is capable of posture control and goal-directed locomotion across uneven terrain. The control architecture is a hierarchical network of simple negative feedback control systems inspired by the organization of the vertebrate nervous system. This robot is capable of robust posture control and locomotion in novel environments with unpredictable disturbances. Unlike current robots, our robot does not use internal inverse and forward models, nor does it require any training in order to perform successfully in novel environments.

Dysfunctional sociability is a core symptom in autism spectrum disorder (ASD) that may arise from neural-network dysconnectivity between multiple brain regions. However, pathogenic neural-network mechanisms underlying social dysfunction are largely unknown. Here, we demonstrate that circuit-selective mutation (ctMUT) of ASD-risk Shank3 gene within a unidirectional projection from the prefrontal cortex to the basolateral amygdala alters spine morphology and excitatory-inhibitory balance of the circuit. Shank3 ctMUT mice show reduced sociability as well as elevated neural activity and its amplitude variability, which is consistent with the neuroimaging results from human ASD patients. Moreover, the circuit hyper-activity disrupts the temporal correlation of socially tuned neurons to the events of social interactions. Finally, optogenetic circuit activation in wild-type mice partially recapitulates the reduced sociability of Shank3 ctMUT mice, while circuit inhibition in Shank3 ctMUT mice partially rescues social behavior. Collectively, these results highlight a circuit-level pathogenic mechanism of Shank3 mutation that drives social dysfunction.

Animals can learn to repeat behaviors to earn desired rewards, a process commonly known as reinforcement learning. While previous work has implicated the ascending dopaminergic projections to the basal ganglia in reinforcement learning, little is known about the role of the hippocampus. Here, we report that a specific population of hippocampal neurons and their dopaminergic innervation contribute to operant self-stimulation. These neurons are located in the dentate gyrus, receive dopaminergic projections from the locus coeruleus, and express D1 dopamine receptors. Activation of D1 + dentate neurons is sufficient for self-stimulation: mice will press a lever to earn optogenetic activation of these neurons. A similar effect is also observed with selective activation of the locus coeruleus projections to the dentate gyrus, and blocked by D1 receptor antagonism. Calcium imaging of D1 + dentate neurons revealed significant activity at the time of action selection, but not during passive reward delivery. These results reveal the role of dopaminergic innervation of the dentate gyrus in supporting operant reinforcement.

Synaptogenesis is essential for circuit development; however, it is unknown whether it is critical for the establishment and performance of goal-directed voluntary behaviors. Here, we show that operant conditioning via lever-press for food reward training in mice induces excitatory synapse formation onto a subset of anterior cingulate cortex neurons projecting to the dorsomedial striatum (ACC→DMS). Training-induced synaptogenesis is controlled by the Gabapentin/Thrombospondin receptor α2δ-1, which is an essential neuronal protein for proper intracortical excitatory synaptogenesis. Using germline and conditional knockout mice, we found that deletion of α2δ-1 in the adult ACC→DMS circuit diminishes training-induced excitatory synaptogenesis. Surprisingly, this manipulation does not impact learning but results in a significant increase in effort exertion without affecting sensitivity to reward value or changing contingencies. Bidirectional optogenetic manipulation of ACC→DMS neurons rescues or phenocopies the behaviors of the α2δ-1 cKO mice, highlighting the importance of synaptogenesis within this cortico-striatal circuit in regulating effort exertion.

The role of dopamine in controlling behavior remains poorly understood. In this study we examined licking behavior in an established hyperdopaminergic mouse model-dopamine transporter knockout (DAT KO) mice. DAT KO mice showed higher rates of licking, which is due to increased perseveration of licking in a bout. By contrast, they showed increased individual lick durations, and reduced inter-lick intervals. During extinction, both KO and control mice transiently increased variability in lick pattern generation while reducing licking rate, yet they showed very different behavioral patterns. Control mice gradually increased lick duration as well as variability. By contrast, DAT KO mice exhibited more immediate (within 10 licks) adjustments-an immediate increase in lick duration variability, as well as more rapid extinction. These results suggest that the level of dopamine can modulate the persistence and pattern generation of a highly stereotyped consummatory behavior like licking, as well as new learning in response to changes in environmental feedback. Increased dopamine in DAT KO mice not only increased perseveration of bouts and individual lick duration, but also increased the behavioral variability in response to the extinction contingency and the rate of extinction.

To understand the neural basis of behavior, it is necessary to record brain activity in freely moving animals. Advances in implantable multi-electrode array technology have enabled researchers to record the activity of neuronal ensembles from multiple brain regions. The full potential of this approach is currently limited by reliance on cable tethers, with bundles of wires connecting the implanted electrodes to the data acquisition system while impeding the natural behavior of the animal. To overcome these limitations, here we introduce a multi-channel wireless headstage system designed for small animals such as rats and mice. A variety of single unit and local field potential signals were recorded from the dorsal striatum and substantia nigra in mice and the ventral striatum and prefrontal cortex simultaneously in rats. This wireless system could be interfaced with commercially available data acquisition systems, and the signals obtained were comparable in quality to those acquired using cable tethers. On account of its small size, light weight, and rechargeable battery, this wireless headstage system is suitable for studying the neural basis of natural behavior, eliminating the need for wires, commutators, and other limitations associated with traditional tethered recording systems.

The medial prefrontal cortex (mPFC) and mediodorsal thalamus (MD) together form a thalamocortical circuit that has been implicated in the learning and production of goal-directed actions. In this study we measured neural activity in both regions simultaneously, as rats learned to press a lever to earn food rewards. In both MD and mPFC, instrumental learning was accompanied by dramatic changes in the firing patterns of the neurons, in particular the rapid emergence of single-unit neural activity reflecting the completion of the action and reward delivery. In addition, we observed distinct patterns of changes in the oscillatory LFP response in MD and mPFC. With learning, there was a significant increase in theta band oscillations (6-10 Hz) in the MD, but not in the mPFC. By contrast, gamma band oscillations (40-55 Hz) increased in the mPFC, but not in the MD. Coherence between these two regions also changed with learning: gamma coherence in relation to reward delivery increased, whereas theta coherence did not. Together these results suggest that, as rats learned the instrumental contingency between action and outcome, the emergence of task related neural activity is accompanied by enhanced functional interaction between MD and mPFC in response to the reward feedback.

In two experiments we investigated the effects of elevated dopaminergic tone on instrumental learning and performance using dopamine transporter knockdown (DAT KD) mice. In Experiment 1, we showed that both DAT KD mice and wild-type controls were similarly sensitive to outcome devaluation induced by sensory specific satiety, indicating normal action-outcome learning in both groups. In Experiment 2, we used a Pavlovian-to-instrumental transfer procedure to assess the potentiation of instrumental responding by Pavlovian conditional stimuli (CS). Although during the Pavlovian training phase the DAT KD mice entered the food magazine more frequently in the absence of the CS, when tested later both groups showed outcome-selective PIT. These results suggest that the elevated dopaminergic tone reduced the selectivity of stimulus control over conditioned behavior, but did not affect instrumental learning.

Most adaptive behaviors require precise tracking of targets in space. In pursuit behavior with a moving target, mice use distance to target to guide their own movement continuously. Here, we show that in the sensorimotor striatum, parvalbumin-positive fast-spiking interneurons (FSIs) can represent the distance between self and target during pursuit behavior, while striatal projection neurons (SPNs), which receive FSI projections, can represent self-velocity. FSIs are shown to regulate velocity-related SPN activity during pursuit, so that movement velocity is continuously modulated by distance to target. Moreover, bidirectional manipulation of FSI activity can selectively disrupt performance by increasing or decreasing the self-target distance. Our results reveal a key role of the FSI-SPN interneuron circuit in pursuit behavior and elucidate how this circuit implements distance to velocity transformation required for the critical underlying computation.

Huntington's disease (HD) is caused by an autosomal dominant polyglutamine expansion mutation of Huntingtin (HTT). HD patients suffer from progressive motor, cognitive, and psychiatric impairments, along with significant degeneration of the striatal projection neurons (SPNs) of the striatum. HD is widely accepted to be caused by a toxic gain-of-function of mutant HTT. However, whether loss of HTT function, because of dominant-negative effects of the mutant protein, plays a role in HD and whether HTT is required for SPN health and function are not known. Here, we delete Htt from specific subpopulations of SPNs using the Cre-Lox system and find that SPNs require HTT for motor regulation, synaptic development, cell health, and survival during aging. Our results suggest that loss of HTT function in SPNs could play a critical role in HD pathogenesis.

Optogenetics and calcium imaging can be combined to simultaneously stimulate and record neural activity in vivo. However, this usually requires two-photon microscopes, which are not portable nor affordable. Here we report the design and implementation of a miniaturized one-photon endoscope for performing simultaneous optogenetic stimulation and calcium imaging. By integrating digital micromirrors, the endoscope makes it possible to activate any neuron of choice within the field of view, and to apply arbitrary spatiotemporal patterns of photostimulation while imaging calcium activity. We used the endoscope to image striatal neurons from either the direct pathway or the indirect pathway in freely moving mice while activating any chosen neuron in the field of view. The endoscope also allows for the selection of neurons based on their relationship with specific animal behaviour, and to recreate the behaviour by mimicking the natural neural activity with photostimulation. The miniaturized endoscope may facilitate the study of how neural activity gives rise to behaviour in freely moving animals.

We recorded activity of dopamine (DA) neurons in the substantia nigra pars compacta in unrestrained mice while monitoring their movements with video tracking. Our approach allows an unbiased examination of the continuous relationship between single unit activity and behavior. Although DA neurons show characteristic burst firing following cue or reward presentation, as previously reported, their activity can be explained by the representation of actual movement kinematics. Unlike neighboring pars reticulata GABAergic output neurons, which can represent vector components of position, DA neurons represent vector components of velocity or acceleration. We found neurons related to movements in four directions-up, down, left, right. For horizontal movements, there is significant lateralization of neurons: the left nigra contains more rightward neurons, whereas the right nigra contains more leftward neurons. The relationship between DA activity and movement kinematics was found on both appetitive trials using sucrose and aversive trials using air puff, showing that these neurons belong to a velocity control circuit that can be used for any number of purposes, whether to seek reward or to avoid harm. In support of this conclusion, mimicry of the phasic activation of DA neurons with selective optogenetic stimulation could also generate movements. Contrary to the popular hypothesis that DA neurons encode reward prediction errors, our results suggest that nigrostriatal DA plays an essential role in controlling the kinematics of voluntary movements. We hypothesize that DA signaling implements gain adjustment for adaptive transition control, and describe a new model of the basal ganglia (BG) in which DA functions to adjust the gain of the transition controller. This model has significant implications for our understanding of movement disorders implicating DA and the BG.

The mediodorsal thalamus (MD) is a crucial component of the neural network involved in the learning and generation of goal-directed actions. A series of experiments reported here examined the contributions of MD to the temporal differentiation of reward-guided actions. In Experiment 1, we trained rats on a discrete-trial, fixed-criterion temporal differentiation task, in which only lever presses exceeding a threshold duration value were rewarded. Pre-training MD lesions impaired temporal differentiation of action duration, by increasing the dispersion of the duration distribution. Post-training MD lesions also impaired differentiation, but by reducing the average emitted press durations, thus shifting the distribution without increasing the dispersion. In Experiment 2, we trained rats to space their lever pressing above criterion inter-press-intervals in order to earn rewards. Both pre-training and post-training MD lesions impaired the differentiation of inter-press-intervals. These results show that MD plays an important role in the acquisition and expression of action differentiation.

A major output nucleus of the basal ganglia is the substantia nigra pars reticulata, which sends GABAergic projections to brainstem and thalamic nuclei. The GABAergic (GABA) neurons are reciprocally connected with nearby dopaminergic neurons, which project mainly to the basal ganglia, a set of subcortical nuclei critical for goal-directed behaviors. Here we examined the impact of motivational states on the activity of GABA neurons in the substantia nigra pars reticulata and the neighboring dopaminergic (DA) neurons in the pars compacta. Both types of neurons show short-latency bursts to a cue predicting a food reward. As mice became sated by repeated consumption of food pellets, one class of neurons reduced cue-elicited firing, whereas another class of neurons progressively increased firing. Extinction or pre-feeding just before the test session dramatically reduced the phasic responses and their motivational modulation. These results suggest that signals related to the current motivational state bidirectionally modulate behavior and the magnitude of phasic response of both DA and GABA neurons in the substantia nigra.