Visual Computing

Headed by Rudolf Mößbauer Tenure Track Professor Matthias Nießner, the Visual Computing Lab at TUM is a group of research enthusiasts pushing the state of the art at the intersection of computer vision, computer graphics, and machine learning.

This Focus Group works on the creation and detection of synthetic media, also known as deepfakes. We have already built a large data set of videos that includes different types of face manipulation (FaceForensics++), and we have used it to train effective tools based on deep learning to distinguish between real and fake content. We have also devised few-shot learning strategies to handle scenarios where only a handful of fake examples are available during training. Currently, we are following innovative identity-based approaches where we look at the problem from a different perspective, by focusing on the facial characteristics of a specific identity. We can then reformulate the problem as an authentication task: Is this the person it is claimed to be? Within this framework, we rely not only on classic features based on the biometrics of a specific subject, but also on new features based on emotions. Moreover, we are developing a multimodal approach that relies on both the video and the audio signals. This new approach will help in enhancing generalization with respect to new forms of manipulation and in improving detection performance in realistic scenarios.

For a very successful example of the work of the Visual Computing group, see their project Face2Face – Real-time Face Capture and Reenactment of RGB Videos.

Since 2018, Hans Fischer Senior Fellow Prof. Leonidas Guibas (Stanford University) and Hans Fischer Fellow Dr. Angel Xuan Chang (Eloquent Labs, Simon Fraser University) have joined the Focus Group, followed by Hans Fischer Senior Fellow Prof. Luisa Verdoliva (University of Federico II, Naples) in 2020.
 

TUM-IAS funded doctoral candidates:
Armen Avetisyan, Visual Computing, TUM
Ji Hou, Visual Computing, TUM
Andreas Rössler, Visual Computing, TUM