Cactus: a Digital Tool for Quality Assurance, Education and Evaluation in Surgical Pathology

Abstract

Purpose To examine the factors that affect inter/intra-observer variability for breast cancer grading on histopathology images, to compare the level of agreement between histopathologists for assessment of virtual slides, and to introduce a new application, CACTUS, for the quality assurance of the cancer grading programme. Methods A new web-based tool for image archiving, annotation, distribution and evaluation (CACTUS) was developed. Four pathologists of varying practice experience in surgical pathology took part in three web-based image circulations using CACTUS. All pathologists evaluated the same 50 images, of which ten images were repeated to assess consistency. Reproducibility was evaluated by intra-observer (intraclass correlation coefficient) and inter-observer (kappa statistics) concordance rates and additional analyses. Results All pathologists found the interface of the tool and presentations of the images quick to learn and easy to use. The pathologists were asked to evaluate all of the images using a menu of questions meant to determine their assessment of the Nottingham grade of a series of x40 images of breast carcinoma cases. For each image, they were asked to give their overall grade impression based on the x40 single image. They were then directed to separate menus of questions to assess the principal components of Nottingham grading: mitotic rate, nuclear morphology and glandular morphology. Their consistency in re-evaluating the set of ten images was excellent agreement (> 0.90). While the overall inter-agreement for three sets is moderate (0.50-0.75), the overall intra-agreement between initial grades and mean value of mitosis, tubular and nuclear scores is excellent, even though opinions were based on single, x40 images. The correlation between initial grade and four criteria/features (mitotic rate, nuclear pleomorphism, tubule formation and the mean value of these three criteria) has been calculated to show which feature is important among others or contributes to the actual score more. Tubular and nuclear scores are highly correlated with initial grades. Outliers among observers are identified by finding their distances from each other and computing the clusters. Conclusion This easy to use, easily modifiable, web-based image archiving, distribution and assessment tool can be useful for evaluating agreement amongst pathologists and disseminating anatomical pathology images for teaching. Moreover, it can be used against a gold standard for the purpose of diagnosis or quality assurance.