Training the Mask R-CNN algorithm to perform complex image segmentation
We adapted the convolutional neural network (CNN) model Mask R-CNN [57] to automatically detect bead protrusions in high-resolution images of nematode dendrites (Fig. 2a). The input to Mask R-CNN is a 1-channel grayscale microscopy image (1024 × 1024 × 1), and the output is a set of predicted bead regions consisting of one binary instance mask (1024 × 1024 × 1) per bead, i.e., a pixel has a value of 1 in the mask when it is part of a bead and 0 otherwise. A tiling procedure was employed to adapt Mask R-CNN for use with 2048 × 2048 × 1 microscopy images (see the “Methods” section), since this image size was sufficient to resolve the smallest bead protrusions. The Mask R-CNN architecture first generates regions of interest (ROIs) using a Faster R-CNN model, composed of a residual network (ResNet-101 [58]) and a feature pyramid network [59]. ROIs are then processed with region proposal and ROI align neural network layers to produce an instance segmentation mask for each detected object. In contrast to thresholding-based methods, which only rely on image intensity for predicting segmentations, CNNs automatically learn and then use hierarchical sets of image features directly from the training data without requiring manual feature engineering. Learning features enable relevant local context to be used in making segmentation predictions, e.g., the shape and size of the bead, what a dendrite looks like, and the proximity of beads to dendrites. We leveraged a transfer learning [60] approach in which Mask R-CNN is pre-trained on a large annotated dataset (ImageNet [61]), and then fine-tuned on a dataset of nematode images that we manually annotated.
The Mask R-CNN algorithm requires a training dataset comprised of raw images of PVD and their corresponding ground truth masks that label the protrusions. The masks were created from raw images using a custom MATLAB code that allows the user to draw around each bead location. A total of 19 images (each with ~ 50–150 beads with an average size of ~ 150 pixels) were manually segmented to compile the training set. In addition, an independent test set was generated with 12 raw images and their associated binary masks. The test set includes diverse images with ~ 30 to ~ 150 beads. These were equally split into images with a low (< 100) and a high (≥ 100) number of beads, to test segmentation consistency. To assess segmentation performance, we quantified precision and recall, described as:
$$ {\displaystyle \begin{array}{l}\mathrm{Precision}=\frac{\mathrm{True}\kern0.17em \mathrm{Positive}}{\mathrm{True}\kern0.17em \mathrm{Positive}+\mathrm{False}\kern0.17em \mathrm{Positive}}\\ {}\mathrm{Recall}=\frac{\mathrm{True}\kern0.17em \mathrm{Positive}}{\mathrm{True}\kern0.17em \mathrm{Positive}+\mathrm{False}\kern0.17em \mathrm{Negative}}\end{array}} $$
In these expressions, true positives are correctly identified beads, false positives are non-bead objects identified as beads, and false negatives are non-identified beads (Fig. 2b). As shown in Fig. 2c, the segmentation precision for the test dataset was 85% and 91% for images with low and high bead numbers, respectively. Similarly, a recall of 90% and 93% was obtained for low and high bead number images, respectively (Fig. 2d). These slight differences could stem from the low number of beads while retaining the same level of objects that can be falsely identified as beads in the first group. The optimized Mask R-CNN algorithm successfully scored 88% in precision and 91% in recall for the entire test set. Thus, this machine learning approach offers consistent unbiased segmentation with high accuracy. To measure the algorithm’s pixel-level accuracy, we calculated the Jaccard index (i.e., intersection over union) of each individual bead in the validation set. The average Jaccard index for all beads was 0.7 (std. dev = 0.15). Furthermore, we show that this index is consistent for animals with drastic beading and with minor beading, as shown in Additional file 1: Fig. S10.
Importantly, precision and recall do not provide information to assess the performance of the model in ignoring objects that can easily be identified as beads (true negatives). In this particular phenotyping problem, this type of objects is prevalent. Autofluorescent lipid droplets can be easily mistaken for neurite protrusions, due to their round shape and location, which can overlap with PVD dendrites in maximum projections. Distinguishing round objects with comparable intensity levels and with similar locations and sizes is a significant challenge. To assess the power of the algorithm to distinguish between the two, we chose 3 images from animals with an abundance of fat droplets that overlapped with dendrites, as part of our training set. As shown in Fig. 2e and Additional file 1: Fig. S1, the algorithm achieved ~ 99% precision in discerning fat droplets from beads, despite their similarities. Prior approaches have addressed this problem by performing dual color microscopy to compare images that show only lipid droplets with images that show the fluorescent reporter [43]. This deep learning approach eliminates the need to perform alternative analyses or dual color microscopy to subtract autofluorescent objects.
Deep phenotyping of age induced PVD neurodegeneration
The nervous system in C. elegans undergoes morphological and functional decline due to aging [14, 18]. Morphological changes in PVD include dendritic outgrowth and beading, which become more common as animals age, as evidenced in Fig. 3a. As previously mentioned, quantitatively investigating beading is difficult as animals can exhibit tens to hundreds of beads with fluorescence intensity levels similar to those of labeled neurons and autofluorescent lipid droplets. Moreover, beading is a highly variable process, and quantification thus requires analysis of large animal populations. We first aimed to quantitatively analyze aging-induced beading in PVD using the deep learning pipeline. Metrics such as average number of beads, size, and inter-bead distance were selected for deeper independent analysis, due to their potential biological significance. These metrics enabled us to examine the morphological changes in PVD. These parameters offered the most descriptive measures which facilitated visualizing the dendritic changes of PVD neuron. Our results (Fig. 3b) show that the average bead count increases from days 2 to 4, 6, and 8 of adulthood. Interestingly, the average number of protrusions does not appear to change significantly afterwards. These results suggest that there may be a saturation point for the beading process, which animals reach at mid-age.
One of the advantages of computer-based image segmentation is that quantification of beading neurodegeneration is not limited to the number of beads. Our post-segmentation MATLAB pipeline enabled extracting additional metrics (a total of 46, Additional file 1) to comprehensively describe the morphological neurodegeneration phenotypes. The average bead size (Fig. 3c) seems to decrease slightly as animals age (days 6–12 vs. days 2–4), which can be explained by an increase in percentage of small beads (area < 100 pixels) (Additional file 1: Fig. S2b). While the size is slightly reduced, the total area occupied by beads increases as nematodes age (Additional file 1: Fig. S2a). These results suggest that the main morphological change induced by aging is an increase in total beading (as measured by number or total bead area), rather than in bead size. The average inter-bead distance (i.e., average of all pairwise distances), which describes how dispersed the beads are, decreases in older populations as expected due to an increase in total number of beads (Fig. 3d). Other metrics that describe bead size and spatial bead distribution (such as 90th percentile of bead size, and percentage of pairwise inter-bead distances < 300 pixels, Additional file 1: Fig. S2d-e) confirmed an overall trend towards accumulation of smaller beads with increased density throughout the neuron in older animals.
To deepen our understanding of aging-induced beading, we compared the patterns exhibited anterior (towards the head) and posterior (towards the tail) to the PVD cell body, since separate images were acquired (Fig. 3a). While both regions exhibit an increase in number of beads (Fig. 3e), this change was more drastic in the anterior section. This difference could be explained either by a higher susceptibility to beading or by the fact that the anterior region occupies larger area, since the posterior is closer to the animal’s tail and is thus more tapered. The average inter-bead distance in the posterior region tends to be larger than in the anterior side (Fig. 3f), as would be expected for a reduced number of beads. As shown in Additional file 1: Fig. S2f, bead morphology appears to be homogeneous, as there is no significant difference in anterior vs. posterior average bead size. Metrics such as the percentage of small beads (< 100 pixels) or the percentage of beads with close neighbors (pairwise inter-bead distances < 300 pixels) did not show any significant differences along the two different sections of PVD (Additional file 1: Fig. S2g-h). This deep learning-based analysis corroborates the neuronal beading reported by Lezi et al., while deepening our understanding of the subtle neurodegenerative patterns that result from aging.
Acute cold-shock induces morphological changes in PVD neuron
In addition to sensing harsh touch, PVD acts as a thermosensor activated by cold temperatures [62]. Cold-shock has been previously studied as a stressor for C. elegans [62,63,64,65,66,67,68,69,70,71,72,73]. Robinson and Powell identified that animals can survive short (4 h) exposures to acute cold-chock (2 °C), but longer exposures (24 h) result in death for a fraction of the population [74]. Furthermore, Ohta et al. showed that the pre-cold-shock culture temperature is inversely correlated with survival rate (more animals survive cold-shock if previously cultured at lower temperatures) [75]. While the detrimental effects of cold-shock on nematodes’ survival and PVD’s involvement in responding to cold temperatures have been independently studied, the impact of cold-shock exposure on PVD health has not been investigated. To answer this question, we first tested the effects of exposure to cold-shock on PVD morphology, where we identified the appearance of PVD neurite beading. Thus, we sought to examine the effects of acute cold-shock at 4 °C on the structure of PVD through our deep learning phenotyping pipeline.
To characterize the relation between cold-shock and beading, we first exposed different C. elegans populations to cold-shock for various durations. As shown in Fig. 4a, eggs extracted from gravid hermaphrodites were transferred to NGM plates and cultured at 20 °C until day 2 of adulthood, when pre-cold-shock microscopy was performed. Nematodes were then split into four separate plates and transferred to 4 °C for either 4, 8, 16, or 24 h. Visual inspection of raw images suggested beading increases with longer cold-shock, but is especially evident in populations that were exposed for 16 h or more. Quantitative analysis performed using the trained Mask R-CNN and post-segmentation feature extraction pipeline shows that the number of beads gradually increases with longer periods of cold-shock (Fig. 4b), and is almost doubled after 16 h, as compared to non-exposed animals. Similar to the aging process, beading reaches a saturation point, where no significant change in the number of beads is observed after 16 h. Interestingly, the percentage of small beads (area < 100 pixels) increases after 4 and 8 h of cold-shock, but this effect is not observed after 16 and 24 h (Additional file 1: Fig. S3b). This suggests that new small beads are generated in the first 8 h, resulting in a higher percentage of smaller beads. The drop in percentage of small beads after 16 and 24 h could be due to existing protrusions becoming larger once the number of beads saturate. This fluctuation in percentage of small beads is also reflected in the average size (Fig. 4c), which slightly decreases during the first 8 h of cold-shock and grows after 16 and 24 h. One potential explanation for these observations is that initially new small beads form, but eventually the beading mechanism switches to bead growth rather than bead generation.
Computer-based image processing and quantitative analysis also enabled identifying subtle differences between aging and cold-shock beading patterns. While an increase in bead number was observed in both cases, cold-shock also resulted in an increase in average inter-bead distance (Fig. 4d), in contrast to aging. This counterintuitive result can potentially be explained by the tendency of cold-induced protrusions to form in more distant dendrites (such as 3rd or 4th order branches) of healthy menorahs. With aging, beads are generated evenly throughout the entire neuron, likely as a result of the aging-induced disorganized branching that increases the density of dendrites (where beads are formed) throughout the worm’s body (Fig. 4e). The information extracted from anterior and posterior regions of PVD for nematodes exposed to acute cold-shock shows very similar patterns to aging-induced neurodegeneration (Additional file 1: Fig. S3f-j). Utilizing this deep learning quantitative phenotyping enabled the identification of a previously unknown effect of acute cold-shock on PVD, which is exacerbated with longer exposures. Moreover, this analysis suggests that beading patterns differ for aging and acute cold-shock, suggesting potentially different mechanisms of protrusion formation.
Post-cold-shock recovery can eliminate PVD dendritic protrusions
Given the significant increase in number of dendritic protrusions in PVD upon exposure to acute cold-shock, we next sought to determine its potential for regeneration. To test this hypothesis, we designed experiments to characterize PVD beading patterns after acute cold exposure and following a subsequent period under normal culture conditions (referred to as rehabilitation or recovery). As shown in Fig. 5a, we performed 3 “1-day” rehabilitation regimes at 3 different temperatures, selected to cover the entire physiological range (15, 20, and 25 °C). Given that nematodes’ growth rate and life span depend on culture temperature, we expected the population cultured at 25 °C to show a faster recovery rate than those grown at 15 °C. After exposure to 16 h of acute cold-shock, the average number of beads increased by 100% as compared to pre-cold-shock conditions. After 1 day of rehabilitation, we observed a decrease in the number of dendritic protrusions in all three rehabilitation temperatures (Fig. 5b). As expected, populations cultured at 15 °C and 25 °C had the lowest (~ 30%) and highest (~ 50%) recovery, respectively, suggesting that recovery rate is correlated with growth rate.
In addition to a reduction in number, the average bead size slightly decreases after rehabilitation (Fig. 5c and Additional file 1: Fig. S5). This recovery is corroborated by the total area covered by beads (Additional file 1: Fig. S4a), which increases after cold-shock and decreases in all recovery regimes, indicating that bead formation due to cold-shock is reversible. These results suggest that recovery occurs by both bead elimination and a gradual size reduction. To further understand the spatial patterns of cold-shock bead formation, we also explored inter-bead distances. As previously mentioned (Fig. 4c), the average inter-bead distance increased post-cold-shock, suggesting beads are formed in the farthest dendrites. One-day recovery treatment at all three temperatures reduced this metric (Fig. 5d), suggesting that beads on the farthest dendrites are more prone to disappear post-recovery. As expected, the percentage of beads with close neighbors (inter-bead distance < 300 pixels) decreases with cold-shock and increases after recovery (Additional file 1: Fig. S4d). Taken together, these quantitative features suggest that cold-shock induces the formation of beads, particularly in distal regions (as the inter-bead distance increases), and that subsequent culture at physiological temperatures reverts these changes.
In line with previous findings, the anterior region of PVD exhibits a higher number of beads than the posterior region, post-rehabilitation. However, recovery does not appear to favor either side, as both areas show a reduction of beading post-recovery (Additional file 1: Fig. S4f). Likewise, while the posterior region shows higher inter-bead distances than the anterior region, both exhibit a reduction of inter-bead distance post-recovery (Additional file 1: Fig. S4g). The average bead size, percentage of small beads (area < 100 pixels), and percentage of beads with close neighbors (inter-bead distance < 300 pixels) do not show any significant differences between the anterior and posterior regions, either post-cold-shock or post-recovery (Additional file 1: Fig. S4h-j), for most conditions. This suggests that the propensity of the anterior region to increased beading observed with aging is also observed upon cold-shock and after recovery from cold-shock. Taken together, these results indicate that after acute cold exposure, 1 day recovery at different temperatures can almost completely alleviate the induced morphological changes of PVD neuron. In addition, this data suggests that a more efficient recovery can be achieved by rehabilitation at higher temperatures. Finally, it appears that cold-shock preferentially induces beading in the farthest dendrites, but these are also preferentially removed during recovery.
Pre-cold-shock culture temperature affects the severity of morphological changes
Physiological culture temperature is a key environmental factor that affects development, growth, and life span in poikilotherms, such as C. elegans [66]. Nematodes habituate to imposed environmental conditions, including temperature [75,76,77,78,79]. Previous studies have identified that after a 4 °C of cold-shock, over 85% of animals cultured at 25 °C die, while most animals cultured at 15 °C survive [75]. These findings motivated us to investigate whether the pre-cold-shock culture temperature plays a role in beading process. To test this, 3 parallel cold-shock/recovery experiments at 3 physiological temperatures were conducted (Fig. 6a) where populations were cultured at 15, 20, and 25 °C for ~ 3.5, 2.5, and 1.5 days, respectively. These animals were then exposed to acute cold-shock at 4 °C and subsequently returned for 1 day to their culture temperature. The difference in culture time prior to cold-shock allowed animals to reach the same developmental stage. Based on prior studies where nematodes cultured at lower temperatures prior to cold-shock have a higher survival rate [75], we hypothesized that lower temperatures would result in less severe morphological changes than high temperatures.
Post-cold-shock behavioral analysis revealed that animals grown at 25 °C were the most affected, as they recovered mobility long after transfer to room temperature (30–40 min), while this time was considerably shorter for animals cultured at 15 and 20 °C. Once animals started crawling, based on qualitative observation, nematodes cultured at 25 °C moved significantly slower than those cultured at lower temperatures. This difference could indicate that worms habituated to a higher temperature may undergo a more drastic shock under cold exposure, although the relevance of locomotion as a metric of shock in the context of PVD is undetermined. These observations suggest that a larger temperature gradient between culture and cold-shock results in increased neuronal damage. As shown in Fig. 6b, the number of beads present after cold-shock and rehabilitation confirms this trend. The average number of beads increases post-cold-shock in all samples, with the smallest change for nematodes grown at 15 °C. The mean bead count after cold-shock reaches the same level for samples cultured at 20 and 25 °C, potentially due to beading reaching a saturation point. This upper limit in number of beads was also observed in neurodegeneration caused by aging and in cold-shock exposure for different periods of time. Interestingly, while populations rehabilitated at 15 and 20 °C show a reduction in number of beads, this effect was not present in those recovered at 25 °C. This could be explained by either a delayed or slower regeneration, or an inability to regenerate for animals cultured at 25 °C. Interestingly, in contrast to animals cultured at 15 and 20 °C, the mean bead size appears to slightly decrease after the rehabilitation regime at 25 °C (Fig. 6c), suggesting that recovery at 25 °C does induce some regenerative effect, although this result was not statistically significant. The regeneration results observed in animals cultured at 20 °C and recovered at 25 °C (presented in the previous section) support the idea that regeneration at 25 °C is possible but is likely slower for the population cultured at 25 °C pre-cold-shock. Such delayed regeneration could stem from the more drastic difference between the baseline and cold-shock temperature. Finally, these experiments corroborate that cold-shock-induced beading occurs in the farthest regions of the neuron, as inter-bead distance increases with cold-shock, and is then reduced after rehabilitation for all culture temperatures (Fig. 6d). The percentage of small beads (area < 100 pixels) and the percentage of beads with close neighbors (inter-bead distances < 300 pixels) (Additional file 1: Fig. S6b,d) also show a reversal of the cold-shock exposure effect in all three physiological temperatures.
Consistent with our previous results, the anterior region of PVD showed a higher number of protrusions than the posterior (Additional file 1: Fig. S6f). Both regions recapitulate the trends observed for pre-cold-shock, post-cold-shock, and post-rehabilitation in the entire animal. The anterior region consistently exhibits ~ 20–100% higher number of beads than the posterior, with pre-cold-shocks showing the largest difference. The average bead size does not show differences between these regions (Additional file 1: Fig. S6h). However, similar to previous experiments, the protrusions are more densely distributed in the anterior part, as is expected for a higher number of beads (Additional file 1: Fig. S6g). The results from this assay support our hypothesis that the culture temperature impacts how nematodes respond to acute cold-shock. Animals cultured at 15 °C exhibited the least morphological changes and faster recovery, while those grown 25 °C showed more drastic beading and slower rehabilitation rate. This difference in response indicates that the magnitude of the cold-shock (based on the baseline temperature) correlates with the induced morphological alteration through a yet unknown mechanism.
Predicting biological status using deep quantitative classification
The quantitative analysis of beading induced by aging and cold-shock indicates that the patterns of PVD morphological changes are different. To further investigate the morphological changes observed, we took advantage of the rich information obtained from the Mask R-CNN segmentation and feature extraction pipeline, which includes all 46 metrics. Through visual inspection of the raw images, as well as the quantitative analysis of the beading patterns, it is clear that beading phenotypes cannot be fully described with a single feature, such as number of beads. Furthermore, there is significant variability within a population. As shown in Fig. 7a, a large fraction of aged animals exhibit less than 70 beads, which is considerably lower than the average of the population and is closer to the number of beads for young individuals. Likewise, some young animals showed more than 70 beads, which is significantly higher than the average of the population. The same variability was observed in cold-shock experiments, suggesting that the number of beads does not offer a comprehensive description about biological status of a nematode. Combining two metrics such as number of beads and average bead size still does not provide enough information to distinguish between young and aged adults (Fig. 7a).
Given that beading patterns relay information about the health state of PVD, we reasoned that beading phenotypes could be used to predict the biological state of the animals. To test this hypothesis, we sought to incorporate all 46 metrics extracted from each image in a classification model. In a first attempt, as shown in Fig. 7b, we performed PCA (principal component analysis) on the 46 metrics. Two principal components (PC1 and PC2) explain 46% of the total variance and are unable to accurately differentiate nematodes from these two stages in their life span. Thus, we aimed to test the ability of classification models to distinguish young and old nematodes using the metrics extracted from PVD beading patterns. As shown in Fig. 7c, animals from different groups (e.g., pre- and post-cold-shock) can exhibit very similar beading patterns. Successful predictive models would prove the presence of subtle patterns that can only be described using multiple metrics. We first developed a classification model to distinguish young vs. old adults. To create a labeled training set, data from the posterior side of PVD for worms younger than 4 days old were grouped together while the second class was comprised of information from nematodes older than 4 days old. An independent validation dataset was then generated to test classification accuracy. It should be noted that these two classes are more difficult to distinguish than comparing day 2 vs. day 12 animals (i.e., the youngest vs. the oldest samples). We tested four classification algorithms: subspace discriminant ensemble (SDE), support vector machines (SVMs), logistic regression, and K-nearest neighbors (KNNs). Two models, SDE and SVM, achieved both training and validation accuracies above 80%, with the validation accuracy of SDE reaching 90% (Fig. 7d). For age-based classification, the information acquired from the PVD anterior side was also used to train separate models leading to training and validation accuracies higher than 80% (Additional file 1: Fig. S7a). In addition, the area under curve (AUC) of the receiver operating characteristic (ROC) curve for both anterior and posterior section reached 0.89 and 0.88, respectively (Additional file 1: Fig. S8). These results suggest that age-induced PVD neurodegeneration causes subtle morphological changes that can only be captured using quantitative deep phenotyping. We also sought to analyze the capability of our trained classifiers in correctly classifying aged nematodes with low number of beads. As shown in Additional file 1: Fig. S9, the trained SDE classifier is capable of identifying aged nematodes even when these appear young, based on their low number of beads, suggesting that other metrics are relevant to differentiate distinguish these two groups. To gain some insight regarding the relevant metrics to distinguish these populations, a stepwise logistic regression was performed and 5 metrics were found to be important. These include the following: percentage of beads with area smaller than 100 pixels (metric 16), standard error of mean for inter-bead distance (metric 20), standard deviation of mean bead intensity (metric 35), 90th percentile of bead intensity (metric 37), and 25th percentile of bead intensity. As shown in Additional file 1: Fig. S9, although the data points for young and aged nematodes overlap for metrics 16, 20, and 37, the classifier is capable of achieving 90% accuracy in distinguishing them, suggesting the beading phenotype requires integration of information from multiple metrics. Similarly, we tested models for classifying nematodes exposed to cold-shock from those that did not experience this stressor. The training and validation set for this analysis was comprised of data from cold-shock performed at all three pre-cold-shock temperatures, and as shown in Fig. 7e, ~ 80% classification accuracy was obtained both in training and validation. Since differences between degenerated (i.e., old or cold-shocked) and healthy (young or non-cold-shocked) animals have been shown, it was expected that these populations are distinguishable. However, given the significant variability in each population, the high classification accuracy obtained was surprising and points to consistent phenotypic patterns exhibited upon morphological alteration that are not evident to visual inspection.
To further test the power of our deep phenotyping pipeline, we next investigated potential differences in PVD exhibited upon aging and acute cold-shock. We compiled data from the anterior and posterior part of the PVD from aging and cold-shock assays to generate training and validation sets. As shown in Fig. 7f, the SDE model reaches ~ 90% training and validation accuracy for the anterior and ~ 80% for the posterior regions (Additional file 1: Fig. S7c). This difference in classification accuracy could stem from the anterior part of PVD undergoing stronger beading patterns than the posterior. Notably, these results indicate that these two stressors cause distinct morphological patterns which can be captured by in-depth quantitative analysis. To further elucidate the differences between these two stressors, we sought to identify the metrics used to distinguish these two groups. The training dataset was used to fit a stepwise logistic regression model to identify the metrics most important in the classification process. Interestingly, the most important metrics were average inter-bead distance (metric 17) and percentage of beads with inter-bead distance below 150 pixels (metric 21). These findings are compatible with the observed trends in average inter-bead distance between aging and acute cold-shock. In addition to these two metrics, median bead size (metric 25), max bead size (metric 26), and median bead intensity (metric 32) were among the metrics incorporated by the stepwise logistic regression model. As a last test, we sought to establish whether the differences in bead patterning between the anterior and posterior part of the PVD could be used to classify images of each class. An accuracy of ~ 85–90% was achieved from different models, confirming underlying beading pattern differences between these two regions of the neuron (Additional file 1: Fig. S7d). The developed classification models are a powerful tool to identify potential differences in beading patterns caused by various environmental stressors (cold-shock or aging).