Short course

Roozbeh Mottaghi
Interactive Scene Understanding

In recent years, deep learning models have demonstrated tremendous improvement in core computer vision problems: image classification, object detection, semantic segmentation, 3D estimation, etc. Now, it might be the time to move one step forward and tackle problems that require higher levels of reasoning that those core components cannot handle single-handedly. My focus in this talk will be on three main topics: (1) Physical scene understanding, where the idea is to learn physics to predict the future movements of objects and to better understand the interaction of objects in a scene.
(2) Learning by interaction, where the idea is to train an autonomous agent by interacting with the environment as opposed to using still images or prerecorded videos as in traditional computer vision models. (3) Learning from intelligent agents, where we mount sensors on a dog and try to learn its behavior.