Student Projects & Theses
We have several opportunities for students to work with us on the latest research in Reinforcement Learning.
The projects are listed below, and you are also encouraged to propose your own topic and cooperate with us.
For potential candidates, please send us an email to nrprojects@informatik.uni-freiburg.de
Current Projects
In case none of the spelled-out projects is of interest, feel free to contact us for new projects via the address mentioned above! Please also include your fields of interest and prior knowledge, as it helps us find a suited supervisor.
Data Evaluation on the Intraoperative Heart-Lung Machine in Paediatric and Adult Cardiac Surgery
Position type: wissenschaftliche Hilfskraft/ student research assistant
Description: The Heart Centre at the University Hospital of Freiburg is conducting research to optimise heart-lung machine (HLM) therapy, which is crucial for peri- and post-operative patient care. More than 600 standardised intraoperative data sets are available for evaluation since 2022. In collaboration with cardiovascular technology, paediatric cardiology and informatics, we're looking for a student research assistant to analyse these data. Your role will involve on-site processing at the University Hospital, including attending heart surgery alongside specialists.
More information: Deutsch, English
Contact: Lisa Graf
Status: open
Mitigating Extrapolation Error In Offline Inverse Reinforcement Learning
Description: Offline Inverse Reinforcement Learning aims to learn a reward function and its corresponding policy from previously collected expert demonstrations. Offline (Deep) RL algorithms use neural networks to approximate the true value functions (Q(s,a) or V(s)) and hence are prone to extrapolation error when estimating the value of out-of-distribution states (states that are not seen in the demonstrations). To combat this two common groups of strategies are employed within the offline RL literature: 1) Restrict the policy to stay close to the demonstrations 2) Restrict the value function to have lower values for out-of-distribution states. In the Inverse Reinforcement setting, we have control over the reward function. In this project, we would like to investigate possible modifications to the reward function such that the derived Value functions and Policies behave well for out-of-distribution states. This can be done by applying the offline RL techniques, used for restricting the policy or value function, to the reward function, as well as developing novel methods to achieve this goal.
Contact: Erfan Azad
Status: full
Previous Projects
Context-aware Reinforcement Learning using Time-series Transformer
Description Generalization to different tasks is always a challenge to Reinforcement Learning. Sometimes, a small change in the environment could drastically influence the performance of the policy learned by Reinforcement Learning. We then need to train an agent which can learn the context/environment change by itself and thus, be able to solve a set of similar tasks. Transformer has been widely used in CV and NLP domains. Recently, there are also a few new models designed for time-serial data. In RL, we also often have time-serial data. Therefore, it's potentially beneficial if we use Transformer model to figure out the context of the environment.
Contact Baohe Zhang
Status full
Benchmarking Constrained Reinforcement Learning Algorithms
Description Constrained Reinforcement Learning is developed for solving tasks which have not only the reward function, but also a set of constraints to follow. As a new field, there isn't yet a benchmark that has compared the performance of algorithms in a scientific manner. Therefore, a new benchmark may be a good move to push the community forward. In this project, you would expect to re-implement some constrained RL algorithms and design a new environment with a set of tasks to compare these algorithms.
Contact Baohe Zhang
Status full
Vision Transformers for efficient policy learning
Description Learning policies from raw videos is often infeasible in real world robotics, as current approaches require large amounts of training data. Extracting object keypoints, can make training significantly faster, unlocking a plethora of interesting tasks. However, they currently require specialized pretraining.
Using vision transformers can remove the need for specialized training and thus make the technique widely available.
In this project, the student(s) first evaluate the keypoint quality for state-of-the-art methods and then extend the technique to more challenging situations.
Hands-on policy learning on a real robot is possible and encouraged.
Contact Jan Ole von Hartz
Keypoints for efficient policy learning
Description As in the project above, we use object keypoints to learn policies more efficiently.
In this project, the student(s) combine object keypoints with the novel SAC-GMM algorithm for policy learning on a real robot.
Contact Jan Ole von Hartz
Reinforcement Learning for Spatial Graph Design
Description In this project, the development of a Reinforcement Learning agent for the design of spatial graphs is to be explored (more details).
Status full
Monte Carlo Tree Search for Antibody Design
Description In this project, we want to utilize Monte Carlo Tree Search methods for the design of antibodies in a simulation (more details).
Status full
Uncertainty-driven Offline model-based RL
Description In this project, the development and usage of world-models in combination with uncertainty estimations for offline Reinforcement Learning is to be explored (more details).
State full
Application of Recurrent Neural Network in Autonomous Driving
Description The state observation is sometimes noisy and partially observed in autonomous driving, which is challenging to solve with usual RL architectures. The recurrent neural network (RNN) is a simple and potential representation for this partial observation. In this project, students are encouraged to explore the usage of RNN in autonomous driving applications.
State full
Autoinflammatory Disease Treatment Recommendation
Description In cooperation with the foundation Rhumatismes-Enfants-Suisse , we develop algorithms for autoinflammatory disease treatment recommendation. The project mainly focuses on unsupervised deep learning, and depending on the progress, on basic deep reinforcement learning ( more details ).
Contact Maria Huegle
State full
High-Level Decision Making in Autonomous Driving
Description We develop deep reinforcement learning algorithms for autonomous lane changes using the open-source traffic simulator SUMO . We focus on various aspects, for example on mixed action spaces, constraints and including predictions of traffic participants.
Contact Gabriel Kalweit and Maria Hügle
State full
Machine Learning for Disease Progression Prediction in Rheumatoid Arthritis
Description In cooperation with the University Hospital in Lausanne, we develop algorithms to predict the disease progression in arthritis based on the Swiss Quality Management (SCQM) database, including lab values, medication, clinical data and patient reported outcomes.
Contact Maria Huegle
State full
Unsupervised Skill Learning from Video
Description In his thesis, Markus Merklinger introduces a model to leverage information from multiple label-free demonstrations in order to yield a meaningful embedding for unseen tasks. A distance measure in the learned embedding space can then be used as a reward function within a reinforcement learning system.
Contact Oier Mees and Gabriel Kalweit
Unsupervised Learning for Early Seizure Detection
In cooperation with the Epilepsy Center in Freiburg, we develop unsupervised learning algorithms to detect epileptic seizures based on intracranial EEG (EcoG) data .
Contact Maria Huegle