Cell segmentation is an image processing step in which cells are algorithmically detected, and the pixels that make up the multiplex image data are split up and assigned to specific cells. For segmentation, the input is an image file and the output is a table of cells with associated information for position, size/boundaries, and the values for each biomarker channel for each pixel located within the cell boundary.
The output cell table is essential for any cell-based analysis.
<aside> đź’ˇ Our default segmentation method for fluorescent images is DeepCell with the following parameters:
n = 9 (resolution 0.377 microns per pixel; adjust as necessary)
</aside>DeepCell Segmentation uses a pre-trained machine learning model Mesmer to perform cell segmentation. Importantly, the model was trained on TissueNet, so it works well on multiplexed tissue image data (i.e., fluorescent images such as ones generated by CODEX). It does not work well on H&E, traditional bright field IHC, cell cultures, or anything else that it was not trained on. It always expects to have a nuclear biomarker channel and an optional membrane channel to perform analysis.
DeepCell was developed by the Van Valen lab at Caltech in partnership with the Angelo lab at Stanford.
DeepCell leverages the deep learning model, Mesmer, to perform nuclear and whole cell segmentation on tissue imaging data. Image from N. Greenwald et al., “Whole-cell segmentation of tissue images with human-level performance using large-scale data annotation and deep learning” (2021).
Watershed Segmentation is a classic method for detecting object boundaries such as cell membranes using only the data (i.e., pixel values) from the image. In brief, the pixel intensities are treated like elevation on a contour map. Local minima are identified and treated as “injection points” for an imaginary liquid. This imaginary liquid continues to be injected until liquid from one local minimum encounters liquid from another local minimum, and the object boundaries are drawn where the liquids encounter each other. For fluorescent images, where higher intensities correspond to positive signal, the contour map is inverted compared to the raw pixel intensity.
The Watershed algorithm segments cells based on local minima and maxima. Image from T. Zheng et al., “Research on Distance Transform and Neural Network Lidar Information Sampling Classification-Based Semantic Segmentation of 2D Indoor Room Maps” (2021).
This method tends to over-segment if there are more local extrema in the image than there are cells, something that could happen if the DNA signal is variable within a nucleus. However, there are methods for setting extrema detection to mitigate this. This method is less restrictive than DeepCell in its applications because it does not rely on a training dataset.
StarDist Segmentation uses a trained convolutional neural network (CNN) to perform cell segmentation, treating each cell as a star-convex shape. This method was developed to improve upon other machine learning methods which tended to perform poorly on areas with high cell density.
StarDist Segmentation treats cells as star-convex objects and uses a CNN to perform segmentation. Image from U. Schmidt et al., “Cell Detection with Star-convex Polygons” (2020).