Preservation of local similarity structure is a keychallenge in deep clustering. Many recent deep clus-tering methods therefore use autoencoders to helpguide the model’s neural network towards an em-bedding which is more reflective of the input spacegeometry. However, recent work has shown thatautoencoder-based deep clustering models can sufferfrom objective function mismatch (OFM). In orderto improve the preservation of local similarity struc-ture, while simultaneously having a low OFM, wedevelop a new auxiliary objective function for deepclustering. Our Unsupervised Companion Objective(UCO) encourages a consistent clustering structureat intermediate layers in the network – helping thenetwork learn an embedding which is more reflectiveof the similarity structure in the input space. Sincea clustering-based auxiliary objective has the samegoal as the main clustering objective, it is less proneto introduce objective function mismatch betweenitself and the main objective.  Our experimentsshow that attaching the UCO to a deep clusteringmodel improves the performance of the model, andexhibits a lower OFM, compared to an analogousautoencoder-based model.