Lung cancer is one of the most common cancers and causes the most deaths worldwide. Once lung cancer is suspected, the tumour is removed surgically from the lung for inspection. A millimeter thin slice is cut from the tumour to inspect under a microscope, where it is possible to see each cell and its shape, and to study both the cancer cells that cause the trouble, but also the body’s immune cells and how the body is fighting back. This forms the basis for deciding which type of tumour, how aggressive it is, and how the body itself is responding. Together with additional information about the patient, such as general health status, the best treatment procedure can begin.

The tumour cell inspection is the single most important tool for successful cancer treatment, together with early detection. Advances in chemotherapy and other treatment regimes have done wonders for cancer survival. It has become possible only due to the meticulous work of pathologists inspecting and describing tumour cells. For more than 200 years, the pathologist has carefully adjusted their microscope and narrowed their eyes before hunching over the tumour slice, ready for a potentially life-saving inspection. Today, you can find the average pathologist in a more comfortable position, looking at the same tumour slice on a big screen - thanks to the introduction of digital pathology, where the tumour slices are stored as data files and can be inspected as any other image.

Digital pathology has opened doors to cancer research that we did not even know existed only a few decades ago. Now that the tumour slice is a digital image, artificial intelligence (AI) methods can be used to analyse it, to support and complement pathologists in their work.

In our project pathology and computer science researchers are working in close collaboration. We have taken one of the most promising AI methods for tumour slice inspection and added a potential improvement. The method recognises and highlights both cancer cells and immune cells in the tumour slice, so it can be an important tool for faster and more accurate inspection of tumours.

We have tested this method on several hundred lung cancer tumours and compared AI to that of traditional methods. The results show that AI can be a better tool for immune cell counting in terms of time and effort spent and can therefore work alongside the pathologist for faster diagnosis and better treatment for the patient.

We want other researchers and clinicians to use and replicate our results. We have therefore made our data, AI models and code open. We have also created a web application prototype that can be used to inspect tumour slices using our AI methods.

Machine learning pipeline: https://github.com/uit-hdl/hovernet-pipeline

Dataset and trained models: https://doi.org/10.18710/4YN9SZ

Web server repository: https://github.com/uit-hdl/histology

‍

Increased levels of tumor infiltrating lymphocytes (TILs) in cancer tissue indicate favourable outcomes in many types of cancer. Manual quantification of immune cells is inaccurate and time consuming for pathologists. Our aim is to leverage a computational solution to automatically quantify TILs in whole slide images (WSIs) of standard diagnostic haematoxylin and eosin stained sections (H&E slides) from lung cancer patients. Our approach is to transfer an open source machine learning method for segmentation and classification of nuclei in H&E slides trained on public data to TIL quantification without manual labeling of our data. Our results show that additional augmentation improves model transferability when training on few samples/limited tissue types. Models trained with sufficient samples/tissue types do not benefit from our additional augmentation policy. Further, the resulting TIL quantification correlates to patient prognosis and compares favorably to the current state-of-the-art method for immune cell detection in non-small lung cancer (current standard CD8 cells in DAB stained TMAs HR 0.34 95% CI 0.17-0.68 vs TILs in HE WSIs: HoVer-Net PanNuke Aug Model HR 0.30 95% CI 0.15-0.60, HoVer-Net MoNuSAC Aug model HR 0.27 95% CI 0.14-0.53). Moreover, we implemented a cloud based system to train, deploy and visually inspect machine learning based annotation for H&E slides. Our pragmatic approach bridges the gap between machine learning research, translational clinical research and clinical implementation. However, validation in prospective studies is needed to assert that the method works in a clinical setting.