Norepinephrine and tradeoff between exploitation and exploration
In continuation of the yesterday's topic. Jonathan Cohen from Princeton and U. of Pittsburg claimes that the level of norepinephrine produced by LC (Locus ceruleus) serves as a switch between exploitation and exploration modes.
On monkeys: exploitation mode: low LC activity, phasic mode of firing of LC neurons; exploration mode: high LC activity, tonic mode of firing of LC neurons, a lot of norepinephrine, pupils dilation.
There are projections from frontal cortical areas evaluating usefulness to Locus ceruleus.
The author thinks about this as a trade-off between efficiency (exploitation) and complexity (exploration). He thinks in terms of a model, when LC-norepinephrine system provides annealing-like mechanisms to the dopamine-based reinforcement learning system.
High LC activity corresponds to "high temperature" and high level of noise in the simulating annealing paradigm. The author thinks about the phasic LC mode as selectively favoring responses to task-relevant events and about the tonic Locus ceruleus mode as producing a more generalized enhancement of responding..
On monkeys: exploitation mode: low LC activity, phasic mode of firing of LC neurons; exploration mode: high LC activity, tonic mode of firing of LC neurons, a lot of norepinephrine, pupils dilation.
There are projections from frontal cortical areas evaluating usefulness to Locus ceruleus.
The author thinks about this as a trade-off between efficiency (exploitation) and complexity (exploration). He thinks in terms of a model, when LC-norepinephrine system provides annealing-like mechanisms to the dopamine-based reinforcement learning system.
High LC activity corresponds to "high temperature" and high level of noise in the simulating annealing paradigm. The author thinks about the phasic LC mode as selectively favoring responses to task-relevant events and about the tonic Locus ceruleus mode as producing a more generalized enhancement of responding..