Sequential decision making
Sequential decision making is a concept in control theory and operations research, which involves making a series of decisions over time to optimize an objective function, such as maximizing cumulative rewards or minimizing costs. In this framework, each decision influences subsequent choices and system outcomes, taking into account the current state, available actions, and the probabilistic nature of state transitions.[1] This process is used for modeling and regulation of dynamic systems, especially under uncertainty, and is commonly addressed using methods like Markov decision processes (MDPs) and dynamic programming.[2]
Applications
[edit]Sequential decision making problems arise across a wide range of domains, including robotics and control (e.g., robot locomotion, manipulation, and autonomous navigation), dynamic resource allocation (such as inventory management and vehicle routing), and game playing, where notable successes include AlphaGo and reinforcement learning agents trained to play Atari video games.[3][4] It has also been applied to clinical trial design and dynamic treatment regimes, where early patient responses inform later treatment decisions.[5]
Reinforcement learning and optimal control are two closely related frameworks used to address sequential decision making problems; optimal control originates from control theory and applied mathematics, while reinforcement learning emerged from artificial intelligence, though the two fields have substantially converged in recent methodology.[6]
References
[edit]- ↑ Puterman, Martin L. (1994). Markov decision processes: discrete stochastic dynamic programming. Wiley series in probability and mathematical statistics. Applied probability and statistics section. New York: Wiley. pp. 1–2. ISBN 978-0-471-61977-2.
- ↑ Bellman, Richard (1958-09-01). "Dynamic programming and stochastic control processes". Information and Control. 1 (3): 228–239. Bibcode:1958InfCo...1..228B. doi:10.1016/S0019-9958(58)80003-0. ISSN 0019-9958.
- ↑ Powell, Warren B. (2019). "From Reinforcement Learning to Optimal Control: A unified framework for sequential decisions". arXiv:1912.03513 [cs.LG].
- ↑ Wang, Zhaoan; Li, Junchao; Mohammad, Mahdi; Xiao, Shaoping (2026). "Reinforcement learning with timed constraints for robotics motion planning". arXiv:2601.00087 [cs.RO].
- ↑ Tec, Mauricio; Duan, Yunshan; Müller, Peter (2022). "A Comparative Tutorial of Bayesian Sequential Design and Reinforcement Learning". The American Statistician. 77 (2): 223–233. arXiv:2205.04023. doi:10.1080/00031305.2022.2129787.
- ↑ Ye, Xiaojing (2026). "Mathematical Foundations of Deep Learning". arXiv:2603.18387 [cs.LG].
 | This machine learning–related article is a stub. You can help Wikipedia by adding missing information. |