Seminar: Self-Supervised Learning

Organizer:
Prof. Dr. Abhinav Valada¹

Co-Organizers:
Juana Valeria Hurtado² Dr. Daniele Cattaneo³ Dr. Tim Welschehold⁴ Daniel Honerkamp⁵

Self-supervised learning is an exciting new paradigm that aims to learn representations from data without explicit supervision. Self-supervised learning has emerged as an alternative to classical supervised learning methods that strongly rely on large labeled datasets for training. Creating datasets with sufficient annotated examples is a challenging task, and the process of labeling data is often arduous, expensive, and sometimes even infeasible. Self-supervised learning mitigates this problem by first learning a pretext task exploiting some property of the data and then using the learned semantically rich representations for fine-tuning on the target task. Here's a recent post that explains this in more detail⁶. In this seminar, we will study a selection of state-of-the-art works that propose self-supervised approaches for perception, state-estimation, and control.

Details

Seminar:	Online via Zoom7, Meeting ID: 939 8825 4440, Password: !SSL2020! Places: Max. 12 students
Beginning:	Friday, May 15, 2020, 14.00-16.00 Introduction via Zoom7 and allocation of seminar topics via email Introductory lecture slides can be downloaded here8
Requirements:	Basic knowledge of deep learning or reinforcement learning.
Remarks:	Due to the Corona crisis, the entire seminar will be held online. Topics will be assigned for the seminar via a preference voting. If there are more interested students than places, places will be assigned based on priority suggestions of the HisInOne system and motivation (tested by asking for a short summary of the preferred paper). The date of registration is irrelevant. In particular, we want to avoid that students grab a topic and then leave the seminar. Please have a coarse look at all available papers to make an informed decision before you commit.

Procedure

• Enroll through HISinOne⁹, the course number is 11LE13-7314-M. Registration period for the seminars in HisInOne are from 11.05.2020 00:00 to 20.05.2020 12:00.
• Attend the introductory session on Friday, May 15, 2020, 14.00-16.00, via Zoom⁷, Meeting ID: 939 8825 4440, Password: !SSL2020!
• Students should select three papers out of the list in preference order and fill the information in this form¹⁰
• Places will be assigned based on priority suggestions of HisInOne and motivation of the student by May 24, 2020.
• Students are requested to prepare a 20 minutes talk, write an abstract and a summary.
• The Seminar will be held as a virtual "Blockseminar" in the last week of July.
• The details of the presentation and the slides should be discussed with the supervisor two weeks before the presentation.
• The abstract should be two pages long and is due June 29, 2020. Please use this template¹¹.
• The summary is due two weeks after the presentation and should be seven pages long at maximum (latex, a4wide, 11pt) not counting the bibliography and figures. Significantly longer summaries will not be accepted.
• The final grade is based on the oral presentation, the written abstract, the summary, and participation in the blockseminar.

Topics

1. SuperPoint: Self-Supervised Interest Point Detection and Description¹²
   Supervisor:Dr. Daniele Cattaneo³
2. GANVO: Unsupervised Deep Monocular Visual Odometry and Depth Estimation with Generative Adversarial Networks¹³
   Supervisor:Dr. Daniele Cattaneo³
3. SelFlow: Self-Supervised Learning of Optical Flow¹⁴
   Supervisor:Dr. Daniele Cattaneo³
4. A Simple Framework for Contrastive Learning of Visual Representations¹⁵
   Supervisor:Daniel Honerkamp⁵
5. Grasp2Vec: Learning Object Representations from Self-Supervised Grasping¹⁶
   Supervisor:Daniel Honerkamp⁵
6. Visual Reinforcement Learning with Imagined Goals¹⁷
   Supervisor:Daniel Honerkamp⁵
7. Improving Semantic Segmentation through Spatio-Temporal Consistency Learned from Videos ¹⁸
   Supervisor:Juana Valeria Hurtado²
8. VideoBERT: A Joint Model for Video and Language Representation Learning ¹⁹
   Supervisor:Juana Valeria Hurtado²
9. Self-Supervised Scene De-occlusion ²⁰
   Supervisor:Juana Valeria Hurtado²
10. Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translation²¹
    Supervisor:Dr. Tim Welschehold⁴
11. Unsupervised Perceptual Rewards for Imitation Learning²²
    Supervisor:Dr. Tim Welschehold⁴
12. Time-Contrastive Networks: Self-Supervised Learning from Video²³
    Supervisor:Dr. Tim Welschehold⁴