Computer vision is ubiquitous in modern society. The major task is to classify images or parts of images into different categories. For instance, in medical image analysis, to distinguish between images containing malignant tumors versus images with normal tissue. Or to classify images taken over ice into different categories depending on whether the images show certain types of seals or just background. These are example of tasks of core interest to Visual Intelligence partners. However, the success of computer vision systems relies on vast amounts of manually labeled images to be used in the learning phase of the deep neural networks which perform the classification. A label corresponds to a certain category of objects, and the process is called supervised learning. As the potential uses of computer vision for classification grows dramatically, the availability of such vast labeled data sets for all possible tasks is simply not realistic. However, the availability of vast amounts of unlabeled images is often realistic and in many cases even unavoidable. To be able to exploit the unlabeled data to perform better classification is a holy grail in deep learning research. This paper takes steps in this direction by merging a so-called clustering module into the deep supervised neural network. The clustering module (CM) is also a neural structure and can therefore be incorporated into the learning process, thereby also exploiting the unlabeled images in a totally new way.  We call this new concept, where the CM aids supervised learning, the SuperCM. We show in the paper that the SuperCM obtains much improved classification results on several different types of image data.

‍

The development of semi-supervised learning (SSL) has in recent years largely focused on the development of new consistency regularization or entropy minimization approaches, often resulting in models with complex trainingstrategies to obtain the desired results. In this work, we instead propose a novel approach that explicitly incorporates the underlying clustering assumption in SSL through extending a recently proposed differentiable clustering module. Leveraging annotated data to guide the cluster centroids results in a simple end-to-end trainable deep SSL approach. We demonstrate that the proposed model improves the performance over the supervised-only baseline and show that our framework can be used in conjunction with other SSL methods to further boost their performance.

‍