Achieving Data-efficient Neural Networks with Hybrid Concept-based Models

Presenter: Tobias Aanderaa Opsahl, Masters student, data science, UiO institute of mathematics.

Abstract:  Most datasets used for machine learning consist of a single label per data point, which is used to optimise the model. However, in cases where more information than just the class label is available, would it be possible to train models more efficiently? We introduce two novel neural network architectures that train with both class labels and additional information in the dataset, referred to as concepts. We call these models hybrid concept- based models, since they use both concept predictions and information not interfering with the concepts to predict the class. In order to thoroughly explore their performance, we introduce ConceptShapes, an open and flexible class of datasets with concept labels. Through various experiments, we show that the hybrid concept-based models outperform standard computer vision models and previously proposed concept-based models with respect to performance, especially in sparse data settings. We also introduce an algorithm for performing adversarial concept attacks, where an image is perturbed in a way that does not change a concept-based model’s concept predictions, but changes the class prediction. We argue that this puts the interpretable qualities promised from previously proposed concept-based models into question.

‍

In compliance with GDPR consent requirements, presentations given in a Visual Intelligence context may be recorded with the consent of the speaker. All recordings are edited to remove all faces, names and voices of other participants. Questions and comments by the audience will hence be removed and will not appear in the recording.  With the freely given consent from the speaker, recorded presentation may be posted on the Visual Intelligence YouTube channel.

‍