Seminar: Learning with Limited Supervision

Organizer:
Prof. Dr. Abhinav Valada

Co-Organizers:
José Arce, Jan Ole von Hartz, Adrian Röfer, Niclas Vödisch

Deep learning has become a key enabler of real world autonomous systems. While classical supervised learning methods typically rely on ground truth information, the area autonomous robotics requires less dependence on manual supervision. The research directions of semi- and self-supervised learning instead aim to learn representations without explicit and potentially even manual supervision. Especially, the domain of robot learning requires scaling to large amounts of unlabeled data in a lifelong manner. Self- and semi-supervised learning already have had a significant impact on fields such as perception, state estimation, control, or graph representation learning, thereby making important progress in object manipulation, scene understanding, visual recognition, object tracking, and learning-based control, amongst others. In this seminar, we will study a selection of state-of-the-art works that propose deep learning techniques for tackling various challenges in autonomous systems. In particular, we will analyze contributions in architecture design and learning paradigms in the areas of computer vision, reinforcement learning, imitation learning, and deep graph learning.

Course Information

Details:	Course Number: 11LE13S-7310-M Places: 12 Location: Georges-Köhler-Allee 80, Room Number 00.021
Course Program:	Introduction: 17/04/2024 @ 10:00 How to make a presentation: 21/06/2024 @ 10:00 Block Seminar: 26/07/2024 @ 09:00 - 17:00
Evaluation Program:	Seminar Presentation: 26/07/2024   Summary Due Date: 02/08/2024 @ 23:59 CEST
Requirements:	Basic knowledge of Deep Learning or Reinforcement Learning
Remarks:	Topics will be assigned for the seminar via preference voting. If there are more interested students than places, places will be assigned based on priority suggestions of the HisInOne system and motivation (tested by asking for a short summary of the preferred paper). The date of registration is irrelevant. In particular, we want to avoid that students grab a topic and then leave the seminar. Please have a coarse look at all available papers to make an informed decision before you commit.

Course Material

Google Form:	Topic Selection Form
Slides:	Lecture 1: Introduction Lecture 2: How to Make a Good Presentation
Templates:	Summary Presentation

Additional Information

Enrollment Procedure

• Enroll through HISinOne, the course number is 11LE13S-7310-M.
• The registration period for the seminars are from 15/04/2024 to 22/04/2024.
• Attend the introductory session on 17/04/2024.
• Select three papers from the topic list (see below) and complete this form by 22/04/2024.
• Places will be assigned based on the priority suggestions of HISInOne and the student's motivation on 25/04/2024.

Evaluation Details

• Students are expected to prepare a 20-minute long presentation and draft a summary.
• The seminar will be held as a "Blockseminar" on 26/07/2024
• The slides of your presentation should be discussed with the supervisor two weeks before the Blockseminar.
• The summary should not exceed seven pages (excluding bibliography and images) and is due on 02/08/2024. Significantly longer summaries will not be accepted.
• Ensure you cite all work you use including images and illustrations. Where possible, try to use your own illustrations.
• The final grade is based on the oral presentation, the summary, and participation in the blockseminar.

What should the Summary contain?

The summary should address the following questions:

• What is the paper's main contribution and why is it important?
• How does it relate to other techniques in the literature?
• What are the strengths and weaknesses of the paper?
• What would be some interesting follow-up work? Can you suggest any possible improvements in the proposed methods? Are there any further interesting applications that the authors might have overlooked?

Graded Component Submission

• Save your document as a PDF and directly submit it to your topic supervisor via email.
• The filename should be in the format "FirstName_LastName_X.pdf" where X is the evaluation component (Summary / Presentation).

Topics

1. WayFASTER: a Self-Supervised Traversability Prediction for Increased Navigation Awareness
   Supervisor: José Arce
2. NeSLAM: Neural Implicit Mapping and Self-Supervised Feature Tracking With Depth Completion and Denoising
   Supervisor: José Arce
3. Category-Level 6D Object Pose and Size Estimation using Self-Supervised Deep Prior Deformation Networks
   Supervisor: José Arce
4. Self-Supervised Monocular Depth Estimation From Thermal Images via Adversarial Multi-Spectral Adaptation
   Supervisor: José Arce
5. DINOv2: Learning Robust Visual Features without Supervision
   Supervisor: Niclas Vödisch
6. STEGO: Unsupervised Semantic Segmentation by Distilling Feature Correspondences
   Supervisor: Niclas Vödisch
7. SemiVL: Semi-Supervised Semantic Segmentation with Vision-Language Guidance
   Supervisor: Niclas Vödisch
8. Seal: Segment Any Point Cloud Sequences by Distilling Vision Foundation Models
   Supervisor: Niclas Vödisch
9. Few-Shot In-Context Imitation Learning via Implicit Graph Alignment
   Supervisor: Adrian Röfer
10. You Only Demonstrate Once: Category-Level Manipulation from Single Visual Demonstration
    Supervisor: Adrian Röfer
11. Leveraging SE(3) Equivariance for Self-Supervised Category-Level Object Pose Estimation
    Supervisor: Adrian Röfer
12. USEEK: Unsupervised SE(3)- Equivariant 3D Keypoints for Generalizable Manipulation
    Supervisor: Adrian Röfer
13. SUGAR: Pre-training 3D Visual Representations for Robotics
    Supervisor: Jan Ole von Hartz
14. PoCo: Policy Composition from and for Heterogeneous Robot Learning
    Supervisor: Jan Ole von Hartz
15. Vid2Robot: End-to-end Video-conditioned Policy Learning with Cross-Attention Transformers
    Supervisor: Jan Ole von Hartz
16. 3D Diffusion Policy: Generalizable Visuomotor Policy Learning via Simple 3D Representations
    Supervisor: Jan Ole von Hartz
17. Hierarchical Policy Blending As Optimal Transport
    Supervisor: Jan Ole von Hartz

Questions?

If you have any questions, please direct them to José Arce before the topic allotment, and to your supervisor after you have received your topic.