In our NeurIPS paper, we leverage diversity stemming from models with different hyperparameters. This leads to SotA accuracy and more robust predictions. *Hyper-deep ensembles* expand on deep ensembles by integrating over a larger space of hyperparameters. *Hyper-batch ensembles* expand on efficient methods. Also check out the thread by Dustin Tran.