AI Safety Seminar

General Information

• Organized by:  Prof. Joschka Boedecker, Julien Brosseit
• HISinOne: 11LE13S-7371-M

• Time: Friday, 14:00-15:30, (First session on 23.10.)

• Location: TBA
• Language: English
• Email:ai-safety-seminar@informatik.uni-freiburg.de

Overview

Large language models (LLMs) can now use tools and carry out tasks with less human supervision. This makes it increasingly important to ensure that they act safely and follow their intended goals. The Hugging Face incident in July 2026 shows why: during internal testing with reduced safeguards, OpenAI agents bypassed restrictions and accessed external systems without permission.

This seminar focuses on AI alignment, methods for guiding LLMs to follow human intentions and safety rules. We study how models learn from human feedback, how AI-generated feedback can support training, and how models can learn to apply safety principles. We also examine why these methods sometimes fail, for example when models exploit weaknesses in their reward signals or hide unwanted behavior. By reading and discussing selected research papers, students will learn how current alignment methods work, what they achieve and which challenges remain.

Format

TBA

(Preliminary) Paper List 

TBA

Resources

Poster guideline as pptx or pdf