Atlas record: https://www.aitimeline.dev/en/events/evt-0143/
Research · 1983
Actor and critic
Barto, Sutton and Anderson split learning into two elements: one chooses an action, the other judges how much better a state is than expected.
Why it matters
Reinforcement learning acquired an architecture that turns a rare reward into a signal available at every step.
The task is balancing a pole on a cart, where the failure signal arrives only when it falls. The critic builds its own value function over states and gives the actor an evaluation at every step. It is the direct ancestor of temporal-difference learning, which Sutton formalised in 1988, and of actor-critic schemes in modern algorithms.
Event record
- Event date
- 1983
- Timeline date
- Event date
- Verification
- Sources gathered automatically · September 17, 2026
- Lines
- ID
- evt-0143
IEEE Transactions on Systems, Man, and Cybernetics volume SMC-13, 1983.
Related events
Earlier
- Enables BOXES: balancing a pole 1968links to this record
Barto, Sutton and Anderson in 1983 take the pole-balancing problem «posed by Michie and Chambers to illustrate their boxes algorithm» and compare their system with BOXES; the paper of Michie and Chambers is in their reference list.
- Builds on Some Studies in Machine Learning Using the Game of Checkers 1959
Updating a value from the difference between successive states comes from Samuel's program.
- Builds on Dynamic Programming 1957
The critic implements Bellman's decomposition for sequential decisions.
Later