Module Code - Title:
CS6482
-
DEEP REINFORCEMENT LEARNING
Year Last Offered:
2025/6
Hours Per Week:
Grading Type:
N
Prerequisite Modules:
Rationale and Purpose of the Module:
Deep Reinforcement Learning (RL) has made significant advances in theory
and practice over the last decade. For example, frameworks that learns
to play Atari games from scratch and improves itself to a superhuman
level. Likewise for the games of Go and poker which the Deep RL
frameworks now play competitively at world championship level.
The objective of this module is to equip students with the fundamental
knowledge and techniques necessary to understand/develop/use Deep
Reinforcement Learning. Emphasis will be placed on applications of Deep
Reinforcement Learning.
Syllabus:
1. Foundations of Reinforcement Learning including Markov Decision
Processes (MDPs), Bellman's Optimality equation, rewards, value
functions and policies, and exploration versus exploitation.
2. Classical tabular-based approaches such as Dynamic Programming, Monte
Carlo, and Temporal Difference Methods (TD) with Eligibility Traces. TD
methods such as Sarsa and Q-Learning will be discussed, as well as RL
architectures such as Actor Critic.
3. Gradient decent methods.
4. RL for planning and learning.
5. New Approaches such as Monte Carlo Tree Search (MCTS),Proximal Policy
Optimisation, and others.
6. Applied Deep RL, for example, using Artificial Neural Networks (ANNs)
and Convolutional Neural Networks (CNNs) for generalisation.
Learning Outcomes:
Cognitive (Knowledge, Understanding, Application, Analysis, Evaluation, Synthesis)
At the end of this module students will be able to:
- Formalize problems as Markov Decision Processes
- Demonstrate an understanding of dynamic programming and exploration / exploitation trade-off
- Know how to implement deep reinforcement learning as an efficient solution approach
- Learn new trends in deep reinforcement learning and apply deep reinforcement learning to problems.
Affective (Attitudes and Values)
At the end of this module students will be able to:
- question and demonstrate whether a deep reinforcement learning based solution is effective, given a learning and planning problem
- select the best deep reinforcement learning based solution to address a a given learning and planning problem
Psychomotor (Physical Skills)
N/A
How the Module will be Taught and what will be the Learning Experiences of the Students:
The module will be delivered using a blended learning approach using traditional classroom lectures, on-line labs and tutorials.
Research Findings Incorporated in to the Syllabus (If Relevant):
Prime Texts:
Sutton and Barto (2018)
Reinforcement Learning, 2nd Edition
, MIT press
Bertsekas, Dimitri P. (2019)
Reinforcement learning and optimal control
, Athena Scientific
Lapan, Maxim (2018)
Deep Reinforcement Learning Hands-On: Apply modern RL methods, with deep Q-networks, value iteration, policy gradients, TRPO, AlphaGo Zero and more
, Packt Publishing Ltd
Other Relevant Texts:
Programme(s) in which this Module is Offered:
Semester(s) Module is Offered:
Spring
Module Leader:
j.j.collins@ul.ie