Baidu’s AI researchers have built an algorithm that can spot cancerous tumors in breast tissue using a method that doesn’t rely solely on neural networks.

Convolutional neural networks (CNNs) excel at pattern matching, and thus are useful at detecting potential clusters of cancerous cells in medical scans. However, CNNs don’t work so well on huge high-resolution images, such as body scans.

The networks break up pictures into, say, 256-pixel squares, and process them one by one. This doesn’t scale over very large images, because the software ends up inspecting the material with a very narrow view, block by block, rather than considering the contents of the neighboring squares. This leads to undesirable predictions, such as false positives.

Baidu’s algorithm – detailed in a paper submitted to the Medical Imaging with Deep Learning conference taking place next month in Amsterdam – tries to minimize false positives by using an imaging method that takes into account pixels surrounding each inspected patch. The system uses a technique called a neural conditional random field (NCRF) – a conditional random field (CRF) glued to a CNN.


“Conditional random fields are a class of statistical modeling method often applied in pattern recognition and machine learning and mostly famous for structured prediction,” Yi Li, coauthor of the paper and a research scientist at Chinese tech behemoth Baidu, told The Register on Monday.

“By structure prediction, it means the model not only predict each independent label, but also try to captures the statistical correlations and structures between labels. In our case, the structure is the spatial correlations between neighboring patches within a 3×3 grid.”

First the CNN, built from the ResNet-18 and ResNet-34 models, evaluates a grid of image patches as an input and encodes each patch into a vector. Next, the CRF uses this information to model the image’s properties and any spatial correlations – basically, clues in the pixels surrounding a given patch that that square may contain cancerous cells.

“NCRF is trained to look at massive number of tumor image patches and normal image patches and extract useful visual features to distinguish them,” Li said. “For example, tumor cells tend to be more actively in mitosis stage, and their nucleus look lighter. Compared to typical CNN, when NCRF extracts these useful features, it not only looks at just one image patch, but also its surrounding patches.”

The researchers trained the NCRF system using the Camelyon16 dataset. It contains 160 normal scans and 110 cancerous tumor scans for training, and 81 benign and 49 malignant tumor slides for testing.

Systems are graded based on a free response receiver operating characteristic (FROC) score, the higher the FROC score the better it is at spotting cancerous tumours. The best average FROC score for the NCRF was 0.8096, beating a professional pathologist, who scored 0.7240. These benchmarks should be taken with a pinch of salt, however. The scores are based on a limited dataset, and are compared against a single professional doctor.

The algorithm has been open sourced so other developers can play around with the code. The researchers are also trying to collaborate with hospitals to get their hands on more pathology slides to test their algorithm.

“If our algorithm can still maintain high accuracies on these new and different dataset, we may think about applying it in clinical settings,” Li concluded.