At the last meeting of Sensorimotor Journal Club, we discussed how different sites of the brain (cortex, basal ganglia, and cerebellum) may be the sites of different types of learning (unsupervised, reward-based, and supervised, respectively), and that each of these types of learning may be needed to learn optimal sensorimotor control. Tomorrow, we will begin to look at how the basal ganglia may be involved in reward-based learning. Anne Findlay will present the following paper:
Schultz, W., Apicella, P., & Ljungberg, T. (1993). Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task. Journal of Neuroscience, 13(3), 900-913. (link to pdf of article)
The present investigation had two aims: (1) to study responses of dopamine neurons to stimuli with attentional and motivational significance during several steps of learning a behavioral task, and (2) to study the activity of dopamine neurons during the performance of cognitive tasks known to be impaired after lesions of these neurons. Monkeys that had previously learned a simple reaction time task were trained to perform a spatial delayed response task via two intermediate tasks. During the learning of each new task, a total of 25% of 76 dopamine neurons showed phasic responses to the delivery of primary liquid reward, whereas only 9% of 163 neurons responded to this event once task performance was established. This produced an average population response during but not after learning of each task. Reward responses during learning were significantly more numerous and pronounced in area A10, as compared to areas A8 and A9. Dopamine neurons also showed phasic responses to the two conditioned stimuli. These were the instruction cue, which was the first stimulus in each trial and indicated the target of the upcoming arm movement (58% of 76 neurons during and 44% of 163 neurons after learning), and the trigger stimulus, which was a conditioned incentive stimulus predicting reward and eliciting a saccadic eye movement and an arm reaching movement (38% of neurons during and 40% after learning). None of the dopamine neurons showed sustained activity in the delay between the instruction and trigger stimuli that would resemble the activity of neurons in dopamine terminal areas, such as the striatum and frontal cortex. Thus, dopamine neurons respond phasically to alerting external stimuli with behavioral significance whose detection is crucial for learning and performing delayed response tasks. The lack of sustained activity suggests that dopamine neurons do not encode representational processes, such as working memory, expectation of external stimuli or reward, or preparation of movement. Rather, dopamine neurons are involved with transient changes of impulse activity in basic attentional and motivational processes underlying learning and cognitive behavior.