NeurIPS 2020: Hyperparameter Ensembles.

In our NeurIPS paper, we leverage diversity stemming from models with different hyperparameters. This leads to SotA accuracy and more robust predictions. Hyper-deep ensembles expand on deep ensembles by integrating over a larger space of hyperparameters. Hyper-batch ensembles expand on efficient methods. Also check out the thread by Dustin Tran.

ICML 2020: Bayesian Neural Networks and the Cold Posterior.

In our ICML 2020 paper, we tackle the question: How Good is the Bayes Posterior in Deep Neural Networks Really? We cast doubt on the current understanding of Bayes posteriors in popular deep neural networks. We find that using cold posteriors improves the performance of BNNs but sharply deviate from the Bayesian paradigm. We put forward several hypotheses that could explain cold posteriors and evaluate the hypotheses through experiments.

AISTATS 2020: Automated Augmented Conjugate Inference.

In our AISTATS paper, we propose a method that automatically transforms non-conjugate Gaussian process models into conjugate models. In the transformed model inference is easy, much faster and more stable. No more need for long manual derivations of complete conditional distributions and their expectations (although I enjoy them from time to time).

I joined Google Brain Berlin.

Great news: I started a postdoctoral researcher position at Google Brain Berlin. I’m excited to work on topics around Bayesian deep learning. Stay tuned! :-)

UAI'19 paper accepted

We have a new paper on augmentation variational inference, to appear at UAI 2019 in Tel Aviv. We consider Gaussian Process multi-class classification and propose a new multi-class likelihood which has two benefits: it leads to well-calibrated uncertainty estimates and allows for an efficient conditionally conjugate latent variable augmentation. The code can be found in our Julia GP package AugmentedGaussianProcesses.

NeurIPS'18 Workshop Presentations

This year at NeurIPS in Montreal, I have three workshop papers. Together with my colleagues, I will present at:

Selected Publications

More Publications

Bayesian Neural Network Priors Revisited
V. Fortuin, A. Garriga-Alonso, Florian Wenzel, G. Rätsch, R. Turner, M. v.d. Wilk, L. Aitchison
arXiv, 2021

PDF Code

How Good is the Bayes Posterior in Deep Neural Networks Really?
F. Wenzel*, K. Roth*, B. Veeling*, J. Świątkowski, L. Tran, S. Mandt, J. Snoek, T. Salimans, R. Jenatton, S. Nowozin (* = equal contribution)
ICML, 2020
Oral Presentation (long)

PDF Code Slides Video

Automated Augmented Conjugate Inference for Non-conjugate Gaussian Process Models
T. Galy-Fajou, F. Wenzel, M. Opper
Oral Presentation

PDF Code

Multi-Class Gaussian Process Classification Made Conjugate: Efficient Inference via Data Augmentation
T. Galy-Fajou*, F. Wenzel*, C. Donner, M. Opper (* = equal contribution)
UAI, 2019

PDF Code ArXiv

Efficient Gaussian Process Classification Using Polya-Gamma Data Augmentation
F. Wenzel*, T. Galy-Fajou*, C. Donner, M. Kloft, M. Opper (* = equal contribution)
AAAI, 2019
Oral Presentation

PDF Code ArXiv

Quasi-Monte Carlo Variational Inference
A. Buchholz*, F. Wenzel*, S. Mandt (* = equal contribution)
ICML, 2018
Oral Presentation


Scalable Generalized Dynamic Topic Models
P. Jähnichen*, F. Wenzel*, M. Kloft, S. Mandt (* = equal contribution)

PDF Code ArXiv

Sparse Probit Linear Mixed Model
S. Mandt*, F. Wenzel*, S. Nakajima, J. P. Cunningham, C. Lippert, M. Kloft (* = equal contribution)
Machine Learning, 2017

PDF Code Journal

Bayesian Nonlinear Support Vector Machines for Big Data
F. Wenzel, T. Galy-Fajou, M. Deutsch, M. Kloft
ECML, 2017
Best Student Paper Award Nomination Oral Presentation

PDF Code

Recent & Upcoming Talks

More Talks

Berlin Bayesians Seminar
Apr 28, 2021. Virtual.
Berlin Machine Learning Seminar (BML)
Feb 17, 2021. Virtual.
University of California, Irvine (UCI) / AIML Seminar
Jan 11, 2021. Virtual.
NeurIPS / Short Teaser Talk for Poster
Dec 10, 2020. Virtual.
Microsoft Research Cambridge
Dec 3, 2020. Virtual.
Google Brain
Nov 30, 2020. Virtual.
ETH Zurich
Nov 24, 2020. Virtual.
ICML / Long Oral Conference Track
Jul 14, 2020. Virtual.
Google Brain Zurich
Jul 2, 2019. Zurich, Switzerland.
TU München
Jun 25, 2019. München, Germany.
University of Oxford
Jun 11, 2019. Oxford, UK.
Hasso Plattner Institut
Mar 21, 2019. Potsdam, Germany.


Supervised Students

  • Lorenz Vaitl: Master’s thesis (TU Berlin, 2018)
    • Scalable Inference for Correlated Noise Classification Models
  • Eren Sezener: Lab rotation project (TU Berlin, 2018)
    • Multi-armed bandits and knowledge gradients


  • Probabilistic Machine Learning (Supervision of student projects, TU Berlin, Winter 18 / 19)

Past Courses