Scalable and Robust Deep Learning

Presenter: Ali Ramezani-Kebrya, Associate Professor in the Department of Informatics at the University of Oslo (UiO)

Abstract: Despite its wide applicability, Deep Learning faces significant challenges including scalability, robustness and security, privacy, fairness, and interpretability. The key algorithm underlying deep learning revolution is stochastic gradient descent, which needs to be distributed to handle enormous and possibly sensitive data distributed among multiple owners, such as hospitals and cellphones, without sharing local data. When training deep neural networks on multiple GPUs/clients, communication time required to share stochastic gradients is the main performance bottleneck. Beyond scalability, it is known that deep learning models are vulnerable to natural distribution shifts and various adversarial attacks at training and inference time. We present efficient gradient compression and robust aggregation schemes to significantly accelerate training and enhance security for image classification tasks.

In compliance with GDPR consent requirements, presentations given in a Visual Intelligence context maybe recorded with the consent of the speaker. All recordings are edited to remove all faces, names and voices of other participants. Questions and comments by the audience will hence be removed and will not appear in the recording. With the freely given consent from the speaker, recorded presentation may be posted on the Visual Intelligence YouTube channel.

‍