NeurIPS 2020: Hyperparameter Ensembles.

In our NeurIPS paper, we leverage diversity stemming from models with different hyperparameters. This leads to SotA accuracy and more robust predictions. Hyper-deep ensembles expand on deep ensembles by integrating over a larger space of hyperparameters. Hyper-batch ensembles expand on efficient methods. Also check out the tweet thread by Dustin Tran.

ICML 2020: Bayesian Neural Networks and the Cold Posterior.

In our ICML 2020 paper, we tackle the question: How Good is the Bayes Posterior in Deep Neural Networks Really? We cast doubt on the current understanding of Bayes posteriors in popular deep neural networks. We find that using cold posteriors improves the performance of BNNs but sharply deviate from the Bayesian paradigm. We put forward several hypotheses that could explain cold posteriors and evaluate the hypotheses through experiments.

AISTATS 2020: Automated Augmented Conjugate Inference.

In our AISTATS paper, we propose a method that automatically transforms non-conjugate Gaussian process models into conjugate models. In the transformed model inference is easy, much faster and more stable. No more need for long manual derivations of complete conditional distributions and their expectations (although I enjoy them from time to time).

I joined Google Brain Berlin.

Great news: I started a postdoctoral researcher position at Google Brain Berlin. I’m excited to work on topics around Bayesian deep learning. Stay tuned! :-)

UAI'19 paper accepted

We have a new paper on augmentation variational inference, to appear at UAI 2019 in Tel Aviv. We consider Gaussian Process multi-class classification and propose a new multi-class likelihood which has two benefits: it leads to well-calibrated uncertainty estimates and allows for an efficient conditionally conjugate latent variable augmentation. The code can be found in our Julia GP package AugmentedGaussianProcesses.

NeurIPS'18 Workshop Presentations

This year at NeurIPS in Montreal, I have three workshop papers. Together with my colleagues, I will present at:

Selected Publications

More Publications

How Good is the Bayes Posterior in Deep Neural Networks Really?
F. Wenzel*, K. Roth*, B. Veeling*, J. Świątkowski, L. Tran, S. Mandt, J. Snoek, T. Salimans, R. Jenatton, S. Nowozin (* = equal contribution)
ICML, 2020
Oral Presentation (long)

PDF Code Slides Video

Automated Augmented Conjugate Inference for Non-conjugate Gaussian Process Models
T. Galy-Fajou, F. Wenzel, M. Opper
AISTATS, 2020
Oral Presentation

PDF Code

Multi-Class Gaussian Process Classification Made Conjugate: Efficient Inference via Data Augmentation
T. Galy-Fajou*, F. Wenzel*, C. Donner, M. Opper (* = equal contribution)
UAI, 2019

PDF Code ArXiv

Efficient Gaussian Process Classification Using Polya-Gamma Data Augmentation
F. Wenzel*, T. Galy-Fajou*, C. Donner, M. Kloft, M. Opper (* = equal contribution)
AAAI, 2019
Oral Presentation

PDF Code ArXiv

Quasi-Monte Carlo Variational Inference
A. Buchholz*, F. Wenzel*, S. Mandt (* = equal contribution)
ICML, 2018
Oral Presentation

PDF ArXiv

Scalable Generalized Dynamic Topic Models
P. Jähnichen*, F. Wenzel*, M. Kloft, S. Mandt (* = equal contribution)
AISTATS, 2018

PDF Code ArXiv

Sparse Probit Linear Mixed Model
S. Mandt*, F. Wenzel*, S. Nakajima, J. P. Cunningham, C. Lippert, M. Kloft (* = equal contribution)
Machine Learning, 2017

PDF Code Journal

Bayesian Nonlinear Support Vector Machines for Big Data
F. Wenzel, T. Galy-Fajou, M. Deutsch, M. Kloft
ECML, 2017
Best Student Paper Award Nomination Oral Presentation

PDF Code

Recent & Upcoming Talks

More Talks

ICML / Long Oral Conference Track
Jul 14, 2020. Virtual.
Google Brain Zurich
Jul 2, 2019. Zurich, Switzerland.
TU München
Jun 25, 2019. München, Germany.
University of Oxford
Jun 11, 2019. Oxford, UK.
Hasso Plattner Institut
Mar 21, 2019. Potsdam, Germany.
Uber AI
Feb 11, 2019. San Francisco, USA.
University of Southern California (USC)
Feb 8, 2019. Los Angeles, USA.
University of California, Irvine (UCI)
Feb 6, 2019. Los Angeles, USA.
Conference on Artificial Intelligence (AAAI)
Jan 31, 2019. Honolulu, Hawaii, USA.
Amazon
Sep 3, 2018. Berlin, Germany.
ICML / Oral Conference Track
Jul 11, 2018. Stockholm, Sweden.
Audi / Audi ML Day
Feb 22, 2018. Ingolstadt, Germany.

Teaching

Supervised Students

  • Lorenz Vaitl: Master’s thesis (TU Berlin, 2018)
    • Scalable Inference for Correlated Noise Classification Models
  • Eren Sezener: Lab rotation project (TU Berlin, 2018)
    • Multi-armed bandits and knowledge gradients

Courses

  • Probabilistic Machine Learning (Supervision of student projects, TU Berlin, Winter 18 / 19)

Past Courses