Fig 1

Fig. 1

Conceptual characterization of the tumor immune microenvironment: the first step (left) indicates the histological and multiplexed imaging datasets that derive from human material after tissue preparation, the second step (middle) represents the different state-of-the-art methodologies briefly showcased in this commentary like deep learning (including graph deep learning) and spatial analysis approaches, and the third step (right) shows the inference over the output for clinical predictive or prognostic purposes.

Expand allCollapse all

The tumor microenvironment (TME) contains malignant cells, immune cells, the tumor vasculature, fibroblasts, pericytes, adipocytes, and other cells [1x[1]Valous, N.A., Rojas Moraleda, R., Jäger, D., Zörnig, I., and Halama, N. Interrogating the microenvironmental landscape of tumors with computational image analysis approaches. Semin Immunol. 2020; 48: 101411https://doi.org/10.1016/j.smim.2020.101411

Crossref | Scopus (3)
| Google ScholarSee all References
]. The emerging TME is a complex and continuously evolving entity; a dynamic and reciprocal relationship develops between tumor cells and components of the TME that supports cancer cell survival, local invasion, and metastatic dissemination [2x[2]Anderson, N.M. and Simon, M.C. The tumor microenvironment. Curr Biol. 2020; 30: R921–R925https://doi.org/10.1016/j.cub.2020.06.081

Abstract | Full Text | Full Text PDF | PubMed | Scopus (280)
| Google ScholarSee all References
]. The tumor immune microenvironment (TIME) is a dynamic system that is modulated by the expression and secretion of proteins that contribute to angiogenesis, immune suppression, and the coordination of immune response [3x[3]Deng, L., Lu, D., Bai, Y., Wang, Y., Bu, H., and Zheng, H. Immune profiles of tumor microenvironment and clinical prognosis among women with triple-negative breast cancer. Cancer Epidemiol Biomarkers Prev. 2019; 28: 1977–1985https://doi.org/10.1158/1055-9965.EPI-19-0469

Crossref | PubMed | Scopus (30)
| Google ScholarSee all References
,4x[4]Quail, D.F. and Joyce, J.A. Microenvironmental regulation of tumor progression and metastasis. Nat Med. 2013; 19: 1423–1437https://doi.org/10.1038/nm.3394

Crossref | PubMed | Scopus (4262)
| Google ScholarSee all References
]. The immune contexture (cellular composition of immune compartment) plays a decisive role in shaping tumor immune responses [5x[5]Giraldo, N.A., Sanchez-Salas, R., Peske, J.D., Vano, Y., Becht, E., Petitprez, F., Validire, P., Ingels, A., Cathelineau, X., Fridman, W.H., and Sautès-Fridman, C. The clinical role of the TME in solid cancer. Br J Cancer. 2018; 120: 45–53https://doi.org/10.1038/s41416-018-0327-z

Crossref | PubMed | Scopus (192)
| Google ScholarSee all References
] and on the metastatic process [6x[6]Mlecnik, B., Bindea, G., Kirilovsky, A., Angell, H.K., Obenauf, A.C., Tosolini, M., Church, S.E., Maby, P., Vasaturo, A., Angelova, M., Fredriksen, T., Mauger, S., Waldner, M., Berger, A., Speicher, M.R., Pagès, F., Valge-Archer, V., and Galon, J. The tumor microenvironment and immunoscore are critical determinants of dissemination to distant metastasis. Sci Transl Med. 2016; 8https://doi.org/10.1126/scitranslmed.aad6352 (327ra26)

Crossref | PubMed | Scopus (311)
| Google ScholarSee all References
].

With the growing importance of immunotherapies to treat cancer patients, it has become crucial to be able to decipher the composition and functional orientation of the TIME [7x[7]Petitprez, F., Sun, C.-.M., Lacroix, L., Sautès-Fridman, C., de Reyniès, A., and Fridman, W.H. Quantitative analyses of the tumor microenvironment composition and orientation in the era of precision medicine. Front Oncol. 2018; 8: 390https://doi.org/10.3389/fonc.2018.00390

Crossref | PubMed | Scopus (33)
| Google ScholarSee all References
,8x[8]Palla, G., Fischer, D.S., Regev, A., and Theis, F.J. Spatial components of molecular tissue biology. Nat Biotechnol. 2022; 40: 308–318https://doi.org/10.1038/s41587-021-01182-1

Crossref | PubMed | Scopus (18)
| Google ScholarSee all References
]. Hence, this commentary aims to briefly present recent computational image workflows that advance the spatio-phenotypic analysis of the TIME. A computational workflow is meant to describe a process for computing, where different parts of the process (tasks) are interdependent, i.e. a task can start processing after its predecessors have (partially) completed and where data flows between tasks [9x[9]Crusoe, M.R., Abeln, S., Iosup, A., Amstutz, P., Chilton, J., Tijanić, N., Ménager, H., Soiland-Reyes, S., Gavrilović, B., and Goble, C. The CWL Community. Methods included: standardizing computational reuse and portability with the common workflow language. Commun ACM. 2022; 65: 54–63https://doi.org/10.1145/3486897

Crossref | Scopus (6)
| Google ScholarSee all References
]. At this point, it is important to note that the primary audience of this piece are active researchers and clinicians that work on the interface of computation and translational applications, as well as computational scientists that briefly wish to examine aspects of the current state-of-the-art pertinent to the scope and impact of core methodological approaches in biomedicine.

For the interested reader, Kelloff and Sigman [10x[10]Kelloff, G.J. and Sigman, C.C. Cancer biomarkers: selecting the right drug for the right patient. Nat Rev Drug Discov. 2012; 11: 201–214https://doi.org/10.1038/nrd3651

Crossref | PubMed | Scopus (197)
| Google ScholarSee all References
] wrote a useful perspective article regarding cancer biomarkers and potential molecular targets for treatment. Spatial relationships among biomarkers are essential for a better understanding of the TIME; the emphasis on getting more information from increasingly smaller biopsy material has resulted in the development of advanced multiplex immunohistochemistry (IHC) strategies for target discovery [11x[11]Halse, H., Colebatch, A.J., Petrone, P., Henderson, M.A., Mills, J.K., Snow, H., Westwood, J.A., Sandhu, S., Raleigh, J.M., Behren, A., Cebon, J., Darcy, P.K., Kershaw, M.H., McArthur, G.A., Gyorki, D.E., and Neeson, P.J. Multiplex immunohistochemistry accurately defines the immune context of metastatic melanoma. Sci Rep. 2018; 8: 11158https://doi.org/10.1038/s41598-018-28944-3

Crossref | PubMed | Scopus (57)
| Google ScholarSee all References
,12x[12]Saylor, J., Ma, Z., Goodridge, H.S., Huang, F., Cress, A.E., Pandol, S.J., Shiao, S.L., Vidal, A.C., Wu, L., Nickols, N.G., Gertych, A., and Knudsen, B.S. Spatial mapping of myeloid cells and macrophages by multiplexed tissue staining. Front Immunol. 2018; 9: 2925https://doi.org/10.3389/fimmu.2018.02925

Crossref | PubMed | Scopus (23)
| Google ScholarSee all References
]. Multiplex IHC allows the evaluation of different TIME component subsets on a single slide, consequently elucidating the interactions between spatially proximal immune cell subpopulations and tumor cells [13x[13]Wang, P.-.C., Hu, Z.-.Q., Zhou, S.-.L., Yu, S.-.Y., Mao, L., Su, S., Li, J., Ren, N., and Huang, X.-.W. The spatial distribution of immune cell subpopulations in hepatocellular carcinoma. Cancer Sci. 2021; 113: 423–431https://doi.org/10.1111/cas.15202

Crossref | PubMed | Scopus (0)
| Google ScholarSee all References
]. Moreover, state-of-the-art multiplexed techniques [14x[14]Lewis, S.M., Asselin-Labat, M.-.L., Nguyen, Q., Berthelet, J., Tan, X., Wimmer, V.C., Merino, D., Rogers, K.L., and Naik, S.H. Spatial omics and multiplexed imaging to explore cancer biology. Nat Methods. 2021; 18: 997–1012https://doi.org/10.1038/s41592-021-01203-6

Crossref | PubMed | Scopus (70)
| Google ScholarSee all References
] offer direct insights for signaling events, subcellular localizations, or translational modifications of proteins, and can complement other bulk assays or single-cell omics analyses [15x[15]Hickey, J.W., Neumann, E.K., Radtke, A.J., Camarillo, J.M., Beuschel, R.T., Albanese, A., McDonough, E., Hatler, J., Wiblin, A.E., Fisher, J., Croteau, J., Small, E.C., Sood, A., Caprioli, R.M., Angelo, R.M., Nolan, G.P., Chung, K., Hewitt, S.M., Germain, R.N., Spraggins, J.M., Lundberg, E., Snyder, M.P., Kelleher, N.L., and Saka, S.K. Spatial mapping of protein composition and tissue organization: a primer for multiplexed antibody-based imaging. Nat Methods. 2022; 19: 284–295https://doi.org/10.1038/s41592-021-01316-y

Crossref | PubMed | Scopus (18)
| Google ScholarSee all References
]. From a methodological perspective, Parra [16x[16]Parra, E.R. Methods to determine and analyze the cellular spatial distribution extracted from multiplex immunofluorescence data to understand the tumor microenvironment. Front Mol Biosci. 2021; 8: 668340https://doi.org/10.3389/fmolb.2021.668340

Crossref | Scopus (6)
| Google ScholarSee all References
] presented a practical approach to study the TIME by explaining mathematical and computational approaches that can be used to quantify the spatial distribution of cell phenotypes in tumor tissues using multiplex immunofluorescence data. Furthermore, emerging technologies for characterizing the spatial architecture of the TIME with a higher flux, higher dimensionality, and higher resolution ratio have extensively expanded the horizon of biomedical research, thus delivering more impact and scope [17x[17]Fu, T., Dai, L.-.J., Wu, S.-.Y., Xiao, Y., Ma, D., Jiang, Y.-.Z., and Shao, Z.-.M. Spatial architecture of the immune microenvironment orchestrates tumor immunity and therapeutic response. J Hematol Oncol. 2021; 14: 98https://doi.org/10.1186/s13045-021-01103-4

Crossref | PubMed | Scopus (39)
| Google ScholarSee all References
]. Probe-based in situ imaging [18x[18]Blise, K.E., Sivagnanam, S., Banik, G.L., Coussens, L.M., and Goecks, J. Single-cell spatial architectures associated with clinical outcome in head and neck squamous cell carcinoma. npj Precis Oncol. 2022; 6: 10https://doi.org/10.1038/s41698-022-00253-z

Crossref | PubMed | Scopus (3)
| Google ScholarSee all References
], spatial transcriptomics [19x[19]Hunter, M.V., Moncada, R., Weiss, J.M., Yanai, I., and White, R.M. Spatially resolved transcriptomics reveals the architecture of the tumor-microenvironment interface. Nat Commun. 2021; 12: 6278https://doi.org/10.1038/s41467-021-26614-z

Crossref | PubMed | Scopus (27)
| Google ScholarSee all References
,20x[20]Larsson, L., Bergenstråhle, L., He, M., Andrusivova, Z., and Lundeberg, J. Snapshot: spatial transcriptomics. Cell. 2022; 185https://doi.org/10.1016/j.cell.2022.06.002 (2840-2840.e1)

Abstract | Full Text | Full Text PDF | PubMed | Scopus (0)
| Google ScholarSee all References
], and spatial proteomics [21x[21]Lundberg, E. and Borner, G.H.H. Spatial proteomics: a powerful discovery tool for cell biology. Nat Rev Mol Cell Biol. 2019; 20: 285–302https://doi.org/10.1038/s41580-018-0094-y

Crossref | PubMed | Scopus (169)
| Google ScholarSee all References
,22x[22]Mund, A., Brunner, A.-.D., and Mann, M. Unbiased spatial proteomics with single-cell resolution in tissues. Mol Cell. 2022; 82: 2335–2349https://doi.org/10.1016/j.molcel.2022.05.022

Abstract | Full Text | Full Text PDF | PubMed | Scopus (1)
| Google ScholarSee all References
] improve our understanding of the complexity and role of the TIME in tumor biology and further facilitate mechanistic research [17x[17]Fu, T., Dai, L.-.J., Wu, S.-.Y., Xiao, Y., Ma, D., Jiang, Y.-.Z., and Shao, Z.-.M. Spatial architecture of the immune microenvironment orchestrates tumor immunity and therapeutic response. J Hematol Oncol. 2021; 14: 98https://doi.org/10.1186/s13045-021-01103-4

Crossref | PubMed | Scopus (39)
| Google ScholarSee all References
]. In this context, Patwa et al. [23x[23]Patwa, A., Yamashita, R., Long, J., Risom, T., Angelo, M., Keren, L., and Rubin, D.L. Multiplexed imaging analysis of the tumor-immune microenvironment reveals predictors of outcome in triple-negative breast cancer. Commun Biol. 2021; 4: 852https://doi.org/10.1038/s42003-021-02361-1

Crossref | PubMed | Scopus (5)
| Google ScholarSee all References
] presented a computational workflow for interrogating multiple features of the TIME, thus demonstrating the potential of cell-to-cell interactions and protein co-expression in improving prognosis and patient stratification. In a more detailed setting, Kuett et al. [24x[24] (Cancer Grand Challenges IMAXT Consortium)Kuett, L., Catena, R., Özcan, A., Plüss, A., Schraml, P., Moch, H., de Souza, N., and Bodenmiller, B. Three-dimensional imaging mass cytometry for highly multiplexed molecular and cellular mapping of tissues and the tumor microenvironment. Nat Cancer. 2021; 3: 122–133https://doi.org/10.1038/s43018-021-00301-w

Crossref | PubMed | Scopus (16)
| Google ScholarSee all References
] showed a multiplexed, single-cell resolution, 3D analysis strategy for FFPE tissues using imaging mass cytometry data. As 3D models reveal cellular and microenvironmental heterogeneity and cell-level tissue organization that is not detectable in 2D representations, this approach could prove beneficial for studying the effect of the microenvironment on tumor cell biology [24x[24] (Cancer Grand Challenges IMAXT Consortium)Kuett, L., Catena, R., Özcan, A., Plüss, A., Schraml, P., Moch, H., de Souza, N., and Bodenmiller, B. Three-dimensional imaging mass cytometry for highly multiplexed molecular and cellular mapping of tissues and the tumor microenvironment. Nat Cancer. 2021; 3: 122–133https://doi.org/10.1038/s43018-021-00301-w

Crossref | PubMed | Scopus (16)
| Google ScholarSee all References
].

Coupling machine learning techniques with new ways to measure the TIME will strengthen our understanding of its role in directing tumor response to therapies [25x[25]Clauset, A., Behbakht, K., and Bitler, B.G. Decoding the dynamic tumor microenvironment. Sci Adv. 2021; 7: eabi5904https://doi.org/10.1126/2Fsciadv.abi5904

Crossref
| Google ScholarSee all References
]. Echle et al. [26x[26]Echle, A., Rindtorff, N.T., Brinker, T.J., Luedde, T., Pearson, A.T., and Kather, J.N. Deep learning in cancer pathology: a new generation of clinical biomarkers. Br J Cancer. 2021; 124: 686–696https://doi.org/10.1038/s41416-020-01122-x

Crossref | PubMed | Scopus (102)
| Google ScholarSee all References
] reviewed applications of deep learning as a framework that can provide clinicians with additional information, thus constituting a new class of biomarkers with potential prognostic and/or predictive power. The use of deep learning methods for quantifying miscellaneous aspects of the TIME [27x[27]Mund, A., Coscia, F., Kriston, A., Hollandi, R., Kovács, F., Brunner, A.-.D., Migh, E., Schweizer, L., Santos, A., Bzorek, M., Naimy, S., Rahbek-Gjerdrum, L.M., Dyring-Andersen, B., Bulkescher, J., Lukas, C., Eckert, M.A., Lengyel, E., Gnann, C., Lundberg, E., Horvath, P., and Mann, M. Deep visual proteomics defines single-cell identity and heterogeneity. Nat Biotechnol. 2022; https://doi.org/10.1038/s41587-022-01302-5

Crossref | PubMed | Scopus (12)
| Google ScholarSee all References
] creates a lot of potential for personalized and stratified approaches [28x[28]Lu, Z., Xu, S., Shao, W., Wu, Y., Zhang, J., Han, Z., Feng, Q., and Huang, K. Deep-learning–based characterization of tumor-infiltrating lymphocytes in breast cancers from histopathology images and multiomics data. JCO Clin Cancer Inform. 2020; 4: 480–490http://ascopubs.org/doi/full/10.1200/CCI.19.00126.

Crossref | PubMed | Scopus (18)
| Google ScholarSee all References
,29x[29]Wang, X., Barrera, C., Bera, K., Viswanathan, V.S., Azarianpour-Esfahani, S., Koyuncu, C., Velu, P., Feldman, M.D., Yang, M., Fu, P., Schalper, K.A., Mahdi, H., Lu, C., Velcheti, V., and Madabhushi, A. Spatial interplay patterns of cancer nuclei and tumor-infiltrating lymphocytes (TILs) predict clinical benefit for immune checkpoint inhibitors. Sci Adv. 2022; 8: eabn3966https://doi.org/10.1126/sciadv.abn3966

Crossref | Scopus (2)
| Google ScholarSee all References
], but the limitations need to be better understood and characterized [30x[30]Liu, X., Faes, L., Kale, A.U., Wagner, S.K., Fu, D.J., Bruynseels, A., Mahendiran, T., Moraes, G., Shamdas, M., Kern, C., Ledsam, J.R., Schmid, M.K., Balaskas, K., Topol, E.J., Bachmann, L.M., Keane, P.A., and Denniston, A.K. A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: a systematic review and meta-analysis. Lancet Digit Health. 2019; 1: e271–e297https://doi.org/10.1016/s2589-7500(19)30123-2

Abstract | Full Text | Full Text PDF | PubMed | Scopus (0)
| Google ScholarSee all References
,31x[31]Xu, H., Cong, F., and Hwang, T.H. Machine learning and artificial intelligence–driven spatial analysis of the tumor immune microenvironment in pathology slides. Eur Urol Focus. 2021; 7: 706–709https://doi.org/10.1016/j.euf.2021.07.006

Abstract | Full Text | Full Text PDF | PubMed | Scopus (2)
| Google ScholarSee all References
]. For example, biomedical experts are interested in understanding the performance of these methods under uncertainty, thus one vital strategy to increase trust in deep learning models is to apply uncertainty quantification methods [32x[32]Abdar, M., Khosravi, A., and Islam. U. Rajendra Acharya, S.M.S. The need for quantification of uncertainty in artificial intelligence for clinical data analysis: increasing the level of trust in the decision-making process. IEEE Syst Man Cybern Mag. 2022; 8: 28–40https://doi.org/10.1109/MSMC.2022.3150144

Crossref | Scopus (0)
| Google ScholarSee all References
]. Another example is the low interpretability of deep neural networks that could prevent their wide application in the clinical setting, hence interpretable approaches need to play a significant role in helping clinicians make treatment decisions and in assessing therapeutic benefit [29x[29]Wang, X., Barrera, C., Bera, K., Viswanathan, V.S., Azarianpour-Esfahani, S., Koyuncu, C., Velu, P., Feldman, M.D., Yang, M., Fu, P., Schalper, K.A., Mahdi, H., Lu, C., Velcheti, V., and Madabhushi, A. Spatial interplay patterns of cancer nuclei and tumor-infiltrating lymphocytes (TILs) predict clinical benefit for immune checkpoint inhibitors. Sci Adv. 2022; 8: eabn3966https://doi.org/10.1126/sciadv.abn3966

Crossref | Scopus (2)
| Google ScholarSee all References
]. In this light, Pour et al. [33x[33]Pour, A.F., White, B.S., Park, J., Sheridan, T.B., and Chuang, J.H. Deep learning features encode interpretable morphologies within histological images. Sci Rep. 2022; 12: 9428https://doi.org/10.1038/s41598-022-13541-2

Crossref | PubMed | Scopus (1)
| Google ScholarSee all References
] investigated the interpretability of H&E-derived convolutional neural network features, confirming that these features encode distinct and identifiable morphology by computationally deconstructing cancer images into interpretable biological features. Tissue images typically comprise a combination of complex sets of patterns [1x[1]Valous, N.A., Rojas Moraleda, R., Jäger, D., Zörnig, I., and Halama, N. Interrogating the microenvironmental landscape of tumors with computational image analysis approaches. Semin Immunol. 2020; 48: 101411https://doi.org/10.1016/j.smim.2020.101411

Crossref | Scopus (3)
| Google ScholarSee all References
] that may exhibit scale-dependent changes in structure [34x[34]Valous, N.A., Lahrmann, B., Halama, N., Bergmann, F., Jäger, D., and Grabe, N. Spatial intratumoral heterogeneity of proliferation in immunohistochemical images of solid tumors. Med Phys. 2016; 43: 2936–2947https://doi.org/10.1118/1.4949003

Crossref | PubMed | Scopus (9)
| Google ScholarSee all References
,35x[35]Valous, N.A., Xiong, W., Halama, N., Zörnig, I., Cantre, D., Wang, Z., Nicolai, B., Verboven, P., and Moraleda, R.Rojas. Multilacunarity as a spatial multiscale multi-mass morphometric of change in the meso-architecture of plant parenchyma tissue. Chaos. 2018; 28: 093110https://doi.org/10.1063/1.5047021

Crossref | PubMed | Scopus (6)
| Google ScholarSee all References
]. Given the different ranges of complexity, the TIME landscape can be quantified in three different scales: image, region-of-interest, and single cell level [36x[36]Lee, K., Lockhart, J.H., Xie, M., Chaudhary, R., Slebos, R.J.C., Flores, E.R., Chung, C.H., and Tan, A.C. Deep learning of histopathology images at the single cell level. Front Artif Intell. 2021; 4: 754641https://doi.org/10.3389/frai.2021.754641

Crossref | Scopus (6)
| Google ScholarSee all References
]. A step forward for single-cell analysis of highly multiplexed imaging data is presented in McKinley et al. [37x[37]McKinley, E.T., Shao, J., Ellis, S.T., Heiser, C.N., Roland, J.T., Macedonia, M.C., Vega, P.N., Shin, S., Coffey, R.J., and Lau, K.S. MIRIAM: a machine and deep learning single-cell segmentation and quantification pipeline for multi-dimensional tissue images. Cytom Part A. 2022; 101: 521–528https://doi.org/10.1002/cyto.a.24541

Crossref | PubMed | Scopus (4)
| Google ScholarSee all References
] where the proposed workflow combined a method for defining cellular compartments, a novel method for extending incomplete cell membranes, and a deep learning-based cell shape descriptor. In a more flexible approach, Din and Yu [38x[38]Din, N.U. and Yu, J. Training a deep learning model for single-cell segmentation without manual annotation. Sci Rep. 2021; 11: 23995https://doi.org/10.1038/s41598-021-03299-4

Crossref | Scopus (0)
| Google ScholarSee all References
] showed that both fluorescence and brightfield microscopy images can be analyzed with a single cell segmentation method without human-labeled data, and only relying on an unsupervised training procedure. In a phenogenomic approach, coupling imaging mass cytometry and genomics, Ali et al. [39x[39]Ali, H.R., Jackson, H.W., Zanotelli, V.R.T., Danenberg, E., Fischer, J.R., Bardwell, H., Provenzano, E., CRUK IMAXT Grand Challenge Team, Rueda, O.M., Chin, S.-.F., Aparicio, S., Caldas, C., and Bodenmiller, B. Imaging mass cytometry and multiplatform genomics define the phenogenomic landscape of breast cancer. Nat Cancer. 2020; 1: 163–175https://doi.org/10.1038/s43018-020-0026-6

Crossref | PubMed | Scopus (90)
| Google ScholarSee all References
] studied the cellular composition of breast tumors while preserving spatial context; the resulting multiplexed tissue maps taken together with genomic data, essentially, linked multidimensional tumor phenotypes with somatic genomic alterations.

Current biomedical and clinical research generates an increasing amount of data; the availability of curated datasets facilitates their interoperability and enables researchers to carry out scientific studies with larger and more connected datasets [40x[40]Legaz-García, M.D.C., Miñarro-Giménez, J.A., Menárguez-Tortosa, M., and Fernández-Breis, J.T. Generation of open biomedical datasets through ontology-driven transformation and integration processes. J Biomed Semantics. 2016; 7: 32https://doi.org/10.1186/s13326-016-0075-z

Crossref | PubMed | Scopus (22)
| Google ScholarSee all References
]. In this milieu of having large volumes of data from multiple (and often heterogeneous) sources, an important aspect of deep neural networks is to be able to train models without explicitly exchanging data among the different sources. With federated learning, models can be trained across several institutions without explicitly sharing patient data [41x[41]Adnan, M., Kalra, S., Cresswell, J.C., Taylor, G.W., and Tizhoosh, H.R. Federated learning and differential privacy for medical image analysis. Sci Rep. 2022; 12: 1953https://doi.org/10.1038/s41598-022-05539-7

Crossref | PubMed | Scopus (8)
| Google ScholarSee all References
], however, a central coordinator governs the learning progress based on all trained models [42x[42]Saldanha, O.L., Quirke, P., West, N.P., James, J.A., Loughrey, M.B., Grabsch, H.I., Salto-Tellez, M., Alwers, E., Cifci, D., Laleh, N.Ghaffari, Seibel, T., Gray, R., Hutchins, G.G.A., Brenner, H., van Treeck, M., Yuan, T., Brinker, T.J., Chang-Claude, J., Khader, F., Schuppert, A., Luedde, T., Trautwein, C., Muti, H.S., Foersch, S., Hoffmeister, M., Truhn, D., and Kather, J.N. Swarm learning for decentralized artificial intelligence in cancer histopathology. Nat Med. 2022; 28: 1232–1239https://doi.org/10.1038/s41591-022-01768-5

Crossref | PubMed | Scopus (10)
| Google ScholarSee all References
]. This limitation of federated learning has been addressed by swarm learning which is a new paradigm [43x[43]Warnat-Herresthal, S., Schultze, H., Shastry, K.L. et al. Swarm learning for decentralized and confidential clinical machine learning. Nature. 2021; 594: 265–270https://doi.org/10.1038/s41586-021-03583-3

Crossref | PubMed | Scopus (105)
| Google ScholarSee all References
] of blockchain-assisted decentralized federated learning [44x[44]Ma. J. Li, C., Shi, L., Ding, M., Wang, T., Han, Z., and Poor, H.V. When federated learning meets blockchain: a new distributed learning paradigm. IEEE Comput Intell Mag. 2022; 17: 26–33https://doi.org/10.1109/MCI.2022.3180932

Crossref | Scopus (2)
| Google ScholarSee all References
]. Saldanha et al. [42x[42]Saldanha, O.L., Quirke, P., West, N.P., James, J.A., Loughrey, M.B., Grabsch, H.I., Salto-Tellez, M., Alwers, E., Cifci, D., Laleh, N.Ghaffari, Seibel, T., Gray, R., Hutchins, G.G.A., Brenner, H., van Treeck, M., Yuan, T., Brinker, T.J., Chang-Claude, J., Khader, F., Schuppert, A., Luedde, T., Trautwein, C., Muti, H.S., Foersch, S., Hoffmeister, M., Truhn, D., and Kather, J.N. Swarm learning for decentralized artificial intelligence in cancer histopathology. Nat Med. 2022; 28: 1232–1239https://doi.org/10.1038/s41591-022-01768-5

Crossref | PubMed | Scopus (10)
| Google ScholarSee all References
] performed a retrospective multicentric study showing that swarm learning-trained models outperformed most locally trained models, and performed on par with models that were trained on merged datasets.

Going beyond the conventional 2D grid representation of images to graph-based representations is becoming increasingly common in the literature [45x[45]Ahmedt-Aristizabal, D., Armin, M.A., Denman, S., Fookes, C., and Petersson, L. Graph-based deep learning for medical diagnosis and analysis: past, present and future. Sensors. 2021; 21: 4758https://doi.org/10.3390/s21144758

Crossref | PubMed | Scopus (14)
| Google ScholarSee all References
]. For example, Li and Gupta [46x[46]Y. Li, A. Gupta. Beyond grids: learning graph representations for visual recognition. In: S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, R. Garnett (editors), Advances in neural information processing systems 31, 2018. https://proceedings.neurips.cc/paper/2018/hash/4efb80f630ccecb2d3b9b2087b0f9c89-Abstract.html.

Google ScholarSee all References
] proposed a method for learning graph representations from an image, where vertices of the graph defined clusters of pixels and the edges measured the similarity between these clusters in a feature space, thus capturing long range dependencies among regions. Graph-based approaches are gaining popularity for the analysis of the TIME, e.g. in Failmezger et al. [47x[47]Failmezger, H., Muralidhar, S., Rullan, A., de Andrea, C.E., Sahai, E., and Yuan, Y. Topological tumor graphs: a graph-based spatial model to infer stromal recruitment for immunosuppression in melanoma histology. Cancer Res. 2020; 80: 1199–1209https://doi.org/10.1158/0008-5472.CAN-19-2268

Crossref | PubMed | Scopus (17)
| Google ScholarSee all References
], a new graph-based method was developed to characterize stromal cell clustering and the stromal barrier between tumor cells and lymphocytes. In a more recent approach, Lu et al. [48x[48]Lu, C., Koyuncu, C., Corredor, G., Prasanna, P., Leo, P., Wang, X.X., Janowczyk, A., Bera, K., Lewis Jr, J., Velcheti, V., and Madabhushi, A. Feature-driven local cell graph (FLocK): new computational pathology-based descriptors for prognosis of lung cancer and HPV status of oropharyngeal cancers. Med Image Anal. 2021; 68: 101903https://doi.org/10.1016/j.media.2020.101903

Abstract | Full Text | Full Text PDF | Scopus (18)
| Google ScholarSee all References
] presented a unique way (different from existing histomorphometrics) to analyze solid tumor images and interrogate tumor morphology by constructing local cell graphs that consider both spatial proximity and attributes of individual nuclei. Furthermore, immune profiling allows us to capture the heterogeneity of the TIME and to gain a mechanistic understanding for predicting clinical responses to treatment as well designing better therapeutic strategies [49x[49]Chuah, S. and Chew, V. High-dimensional immune-profiling in cancer: implications for immunotherapy. J Immunother Cancer. 2020; 8: e000363https://doi.org/10.1136/jitc-2019-000363

Crossref | PubMed | Scopus (32)
| Google ScholarSee all References
]. Apart from enabling a deeper understanding of the local immune response in the TIME, high-dimensional immune profiling has been valuable in screening for immune-related signatures which might serve as potential biomarkers in predicting response to cancer therapy [49x[49]Chuah, S. and Chew, V. High-dimensional immune-profiling in cancer: implications for immunotherapy. J Immunother Cancer. 2020; 8: e000363https://doi.org/10.1136/jitc-2019-000363

Crossref | PubMed | Scopus (32)
| Google ScholarSee all References
]. The potential biomarkers discovered should be considered as hypothesis-generating [50x[50]Biesecker, L.G. Hypothesis-generating research and predictive medicine. Genome Res. 2013; 23: 1051–1053https://doi.org/10.1101/gr.157826.113

Crossref | PubMed | Scopus (49)
| Google ScholarSee all References
], and they need to be validated (analytically and clinically) before adoption [51x[51]Ou, F.-.S., Michiels, S., Shyr, Y., Adjei, A.A., and Oberg, A.L. Biomarker discovery and validation: statistical considerations. J Thorac Oncol. 2021; 16: 537–545https://doi.org/10.1016/j.jtho.2021.01.1616

Abstract | Full Text | Full Text PDF | PubMed | Scopus (16)
| Google ScholarSee all References
]. Graph deep learning approaches [52x[52]Zhang, Z., Cui, P., and Zhu, W. Deep learning on graphs: a survey. IEEE Trans Knowl Data Eng. 2022; 34: 249–270https://doi.org/10.1109/TKDE.2020.2981333

Crossref | Scopus (181)
| Google ScholarSee all References
] are very promising in analyzing image data from multiplexing imaging platforms. Essentially, the graph representation provides an advantage for identifying patterns suitable for predictive/exploratory analysis, and combined with deep neural networks allows for a learning process that takes into account the explicit relations of the data. In this context, Martin et al. [53x[53]N.G. Martin, S. Malacrino, M. Wojciechowska, L. Campo, H. Jones, D.C. Wedge, C. Holmes, K. Sirinukunwattana, H. Sailem, C. Verrill, J. Rittscher. A graph based neural network approach to immune profiling of multiplexed tissue samples. arXiv 2022;2202.00813. https://arxiv.org/abs/2202.00813.

Google ScholarSee all References
] proposed a graph neural network that profiled the microenvironment associated with different tumor stages in an explainable setting.

Fig. 1 conceptualizes the characterization of the TIME with current state-of-the-art methods, briefly showcased in this commentary (e.g. deep learning and spatial analysis approaches), on human histological and multiplexed data in order to construct and validate clinical prognostic or predictive models. In Fig. 1, methodologies are portrayed as a puzzle because in many instances they can be part of complex computational workflows where, e.g. deep learning is used to segment objects (tumor cells, immune cells, etc.) or regions of interest (tumor parenchyma, tumor stroma, etc.), followed by methods that measure spatial organization or interplay among the different components of the TIME.

Fig 1 Opens large image

Fig. 1

Conceptual characterization of the tumor immune microenvironment: the first step (left) indicates the histological and multiplexed imaging datasets that derive from human material after tissue preparation, the second step (middle) represents the different state-of-the-art methodologies briefly showcased in this commentary like deep learning (including graph deep learning) and spatial analysis approaches, and the third step (right) shows the inference over the output for clinical predictive or prognostic purposes.

Novel computational image workflows have potential in dissecting the interplay between malignant cells and the immune and stromal elements that comprise the tumor microenvironment [54x[54]Liu, C.C., Steen, C.B., and Newman, A.M. Computational approaches for characterizing the tumor immune microenvironment. Immunology. 2019; 158: 70–84https://doi.org/10.1111/imm.13101

Crossref | PubMed | Scopus (20)
| Google ScholarSee all References
,55x[55]Jiménez-Sánchez, D., Ariz, M., Chang, H., Matias-Guiu, X., de Andrea, C.E., and Ortiz-de-Solórzano, C. NaroNet: discovery of tumor microenvironment elements from highly multiplexed images. Med Image Anal. 2022; 78: 102384https://doi.org/10.1016/j.media.2022.102384 (])

Abstract | Full Text | Full Text PDF | PubMed | Scopus (5)
| Google ScholarSee all References
]. These workflows will continue to push the limits of the technological envelope, and to provide tangible benefits in terms of novel insights into tumor-immune interactions as well as assisting in clinical decision making.

Declaration of Competing Interest

The authors declare no conflict of interest.

References

  1. [1]Valous, N.A., Rojas Moraleda, R., Jäger, D., Zörnig, I., and Halama, N. Interrogating the microenvironmental landscape of tumors with computational image analysis approaches. Semin Immunol. 2020; 48: 101411https://doi.org/10.1016/j.smim.2020.101411
  2. [2]Anderson, N.M. and Simon, M.C. The tumor microenvironment. Curr Biol. 2020; 30: R921–R925https://doi.org/10.1016/j.cub.2020.06.081
  3. [3]Deng, L., Lu, D., Bai, Y., Wang, Y., Bu, H., and Zheng, H. Immune profiles of tumor microenvironment and clinical prognosis among women with triple-negative breast cancer. Cancer Epidemiol Biomarkers Prev. 2019; 28: 1977–1985https://doi.org/10.1158/1055-9965.EPI-19-0469
  4. [4]Quail, D.F. and Joyce, J.A. Microenvironmental regulation of tumor progression and metastasis. Nat Med. 2013; 19: 1423–1437https://doi.org/10.1038/nm.3394
  5. [5]Giraldo, N.A., Sanchez-Salas, R., Peske, J.D., Vano, Y., Becht, E., Petitprez, F., Validire, P., Ingels, A., Cathelineau, X., Fridman, W.H., and Sautès-Fridman, C. The clinical role of the TME in solid cancer. Br J Cancer. 2018; 120: 45–53https://doi.org/10.1038/s41416-018-0327-z
  6. [6]Mlecnik, B., Bindea, G., Kirilovsky, A., Angell, H.K., Obenauf, A.C., Tosolini, M., Church, S.E., Maby, P., Vasaturo, A., Angelova, M., Fredriksen, T., Mauger, S., Waldner, M., Berger, A., Speicher, M.R., Pagès, F., Valge-Archer, V., and Galon, J. The tumor microenvironment and immunoscore are critical determinants of dissemination to distant metastasis. (327ra26)Sci Transl Med. 2016; 8https://doi.org/10.1126/scitranslmed.aad6352
  7. [7]Petitprez, F., Sun, C.-.M., Lacroix, L., Sautès-Fridman, C., de Reyniès, A., and Fridman, W.H. Quantitative analyses of the tumor microenvironment composition and orientation in the era of precision medicine. Front Oncol. 2018; 8: 390https://doi.org/10.3389/fonc.2018.00390
  8. [8]Palla, G., Fischer, D.S., Regev, A., and Theis, F.J. Spatial components of molecular tissue biology. Nat Biotechnol. 2022; 40: 308–318https://doi.org/10.1038/s41587-021-01182-1
  9. [9]Crusoe, M.R., Abeln, S., Iosup, A., Amstutz, P., Chilton, J., Tijanić, N., Ménager, H., Soiland-Reyes, S., Gavrilović, B., and Goble, C. The CWL Community. Methods included: standardizing computational reuse and portability with the common workflow language. Commun ACM. 2022; 65: 54–63https://doi.org/10.1145/3486897
  10. [10]Kelloff, G.J. and Sigman, C.C. Cancer biomarkers: selecting the right drug for the right patient. Nat Rev Drug Discov. 2012; 11: 201–214https://doi.org/10.1038/nrd3651
  11. [11]Halse, H., Colebatch, A.J., Petrone, P., Henderson, M.A., Mills, J.K., Snow, H., Westwood, J.A., Sandhu, S., Raleigh, J.M., Behren, A., Cebon, J., Darcy, P.K., Kershaw, M.H., McArthur, G.A., Gyorki, D.E., and Neeson, P.J. Multiplex immunohistochemistry accurately defines the immune context of metastatic melanoma. Sci Rep. 2018; 8: 11158https://doi.org/10.1038/s41598-018-28944-3
  12. [12]Saylor, J., Ma, Z., Goodridge, H.S., Huang, F., Cress, A.E., Pandol, S.J., Shiao, S.L., Vidal, A.C., Wu, L., Nickols, N.G., Gertych, A., and Knudsen, B.S. Spatial mapping of myeloid cells and macrophages by multiplexed tissue staining. Front Immunol. 2018; 9: 2925https://doi.org/10.3389/fimmu.2018.02925
  13. [13]Wang, P.-.C., Hu, Z.-.Q., Zhou, S.-.L., Yu, S.-.Y., Mao, L., Su, S., Li, J., Ren, N., and Huang, X.-.W. The spatial distribution of immune cell subpopulations in hepatocellular carcinoma. Cancer Sci. 2021; 113: 423–431https://doi.org/10.1111/cas.15202
  14. [14]Lewis, S.M., Asselin-Labat, M.-.L., Nguyen, Q., Berthelet, J., Tan, X., Wimmer, V.C., Merino, D., Rogers, K.L., and Naik, S.H. Spatial omics and multiplexed imaging to explore cancer biology. Nat Methods. 2021; 18: 997–1012https://doi.org/10.1038/s41592-021-01203-6
  15. [15]Hickey, J.W., Neumann, E.K., Radtke, A.J., Camarillo, J.M., Beuschel, R.T., Albanese, A., McDonough, E., Hatler, J., Wiblin, A.E., Fisher, J., Croteau, J., Small, E.C., Sood, A., Caprioli, R.M., Angelo, R.M., Nolan, G.P., Chung, K., Hewitt, S.M., Germain, R.N., Spraggins, J.M., Lundberg, E., Snyder, M.P., Kelleher, N.L., and Saka, S.K. Spatial mapping of protein composition and tissue organization: a primer for multiplexed antibody-based imaging. Nat Methods. 2022; 19: 284–295https://doi.org/10.1038/s41592-021-01316-y
  16. [16]Parra, E.R. Methods to determine and analyze the cellular spatial distribution extracted from multiplex immunofluorescence data to understand the tumor microenvironment. Front Mol Biosci. 2021; 8: 668340https://doi.org/10.3389/fmolb.2021.668340
  17. [17]Fu, T., Dai, L.-.J., Wu, S.-.Y., Xiao, Y., Ma, D., Jiang, Y.-.Z., and Shao, Z.-.M. Spatial architecture of the immune microenvironment orchestrates tumor immunity and therapeutic response. J Hematol Oncol. 2021; 14: 98https://doi.org/10.1186/s13045-021-01103-4
  18. [18]Blise, K.E., Sivagnanam, S., Banik, G.L., Coussens, L.M., and Goecks, J. Single-cell spatial architectures associated with clinical outcome in head and neck squamous cell carcinoma. npj Precis Oncol. 2022; 6: 10https://doi.org/10.1038/s41698-022-00253-z
  19. [19]Hunter, M.V., Moncada, R., Weiss, J.M., Yanai, I., and White, R.M. Spatially resolved transcriptomics reveals the architecture of the tumor-microenvironment interface. Nat Commun. 2021; 12: 6278https://doi.org/10.1038/s41467-021-26614-z
  20. [20]Larsson, L., Bergenstråhle, L., He, M., Andrusivova, Z., and Lundeberg, J. Snapshot: spatial transcriptomics. (2840-2840.e1)Cell. 2022; 185https://doi.org/10.1016/j.cell.2022.06.002
  21. [21]Lundberg, E. and Borner, G.H.H. Spatial proteomics: a powerful discovery tool for cell biology. Nat Rev Mol Cell Biol. 2019; 20: 285–302https://doi.org/10.1038/s41580-018-0094-y
  22. [22]Mund, A., Brunner, A.-.D., and Mann, M. Unbiased spatial proteomics with single-cell resolution in tissues. Mol Cell. 2022; 82: 2335–2349https://doi.org/10.1016/j.molcel.2022.05.022
  23. [23]Patwa, A., Yamashita, R., Long, J., Risom, T., Angelo, M., Keren, L., and Rubin, D.L. Multiplexed imaging analysis of the tumor-immune microenvironment reveals predictors of outcome in triple-negative breast cancer. Commun Biol. 2021; 4: 852https://doi.org/10.1038/s42003-021-02361-1
  24. [24] (Cancer Grand Challenges IMAXT Consortium)Kuett, L., Catena, R., Özcan, A., Plüss, A., Schraml, P., Moch, H., de Souza, N., and Bodenmiller, B. Three-dimensional imaging mass cytometry for highly multiplexed molecular and cellular mapping of tissues and the tumor microenvironment. Nat Cancer. 2021; 3: 122–133https://doi.org/10.1038/s43018-021-00301-w
  25. [25]Clauset, A., Behbakht, K., and Bitler, B.G. Decoding the dynamic tumor microenvironment. Sci Adv. 2021; 7: eabi5904https://doi.org/10.1126/2Fsciadv.abi5904
  26. [26]Echle, A., Rindtorff, N.T., Brinker, T.J., Luedde, T., Pearson, A.T., and Kather, J.N. Deep learning in cancer pathology: a new generation of clinical biomarkers. Br J Cancer. 2021; 124: 686–696https://doi.org/10.1038/s41416-020-01122-x
  27. [27]Mund, A., Coscia, F., Kriston, A., Hollandi, R., Kovács, F., Brunner, A.-.D., Migh, E., Schweizer, L., Santos, A., Bzorek, M., Naimy, S., Rahbek-Gjerdrum, L.M., Dyring-Andersen, B., Bulkescher, J., Lukas, C., Eckert, M.A., Lengyel, E., Gnann, C., Lundberg, E., Horvath, P., and Mann, M. Deep visual proteomics defines single-cell identity and heterogeneity. Nat Biotechnol. 2022; https://doi.org/10.1038/s41587-022-01302-5
  28. [28]Lu, Z., Xu, S., Shao, W., Wu, Y., Zhang, J., Han, Z., Feng, Q., and Huang, K. Deep-learning–based characterization of tumor-infiltrating lymphocytes in breast cancers from histopathology images and multiomics data. JCO Clin Cancer Inform. 2020; 4: 480–490
  29. [29]Wang, X., Barrera, C., Bera, K., Viswanathan, V.S., Azarianpour-Esfahani, S., Koyuncu, C., Velu, P., Feldman, M.D., Yang, M., Fu, P., Schalper, K.A., Mahdi, H., Lu, C., Velcheti, V., and Madabhushi, A. Spatial interplay patterns of cancer nuclei and tumor-infiltrating lymphocytes (TILs) predict clinical benefit for immune checkpoint inhibitors. Sci Adv. 2022; 8: eabn3966https://doi.org/10.1126/sciadv.abn3966
  30. [30]Liu, X., Faes, L., Kale, A.U., Wagner, S.K., Fu, D.J., Bruynseels, A., Mahendiran, T., Moraes, G., Shamdas, M., Kern, C., Ledsam, J.R., Schmid, M.K., Balaskas, K., Topol, E.J., Bachmann, L.M., Keane, P.A., and Denniston, A.K. A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: a systematic review and meta-analysis. Lancet Digit Health. 2019; 1: e271–e297https://doi.org/10.1016/s2589-7500(19)30123-2
  31. [31]Xu, H., Cong, F., and Hwang, T.H. Machine learning and artificial intelligence–driven spatial analysis of the tumor immune microenvironment in pathology slides. Eur Urol Focus. 2021; 7: 706–709https://doi.org/10.1016/j.euf.2021.07.006
  32. [32]Abdar, M., Khosravi, A., and Islam. U. Rajendra Acharya, S.M.S. The need for quantification of uncertainty in artificial intelligence for clinical data analysis: increasing the level of trust in the decision-making process. IEEE Syst Man Cybern Mag. 2022; 8: 28–40https://doi.org/10.1109/MSMC.2022.3150144
  33. [33]Pour, A.F., White, B.S., Park, J., Sheridan, T.B., and Chuang, J.H. Deep learning features encode interpretable morphologies within histological images. Sci Rep. 2022; 12: 9428https://doi.org/10.1038/s41598-022-13541-2
  34. [34]Valous, N.A., Lahrmann, B., Halama, N., Bergmann, F., Jäger, D., and Grabe, N. Spatial intratumoral heterogeneity of proliferation in immunohistochemical images of solid tumors. Med Phys. 2016; 43: 2936–2947https://doi.org/10.1118/1.4949003
  35. [35]Valous, N.A., Xiong, W., Halama, N., Zörnig, I., Cantre, D., Wang, Z., Nicolai, B., Verboven, P., and Moraleda, R.Rojas. Multilacunarity as a spatial multiscale multi-mass morphometric of change in the meso-architecture of plant parenchyma tissue. Chaos. 2018; 28: 093110https://doi.org/10.1063/1.5047021
  36. [36]Lee, K., Lockhart, J.H., Xie, M., Chaudhary, R., Slebos, R.J.C., Flores, E.R., Chung, C.H., and Tan, A.C. Deep learning of histopathology images at the single cell level. Front Artif Intell. 2021; 4: 754641https://doi.org/10.3389/frai.2021.754641
  37. [37]McKinley, E.T., Shao, J., Ellis, S.T., Heiser, C.N., Roland, J.T., Macedonia, M.C., Vega, P.N., Shin, S., Coffey, R.J., and Lau, K.S. MIRIAM: a machine and deep learning single-cell segmentation and quantification pipeline for multi-dimensional tissue images. Cytom Part A. 2022; 101: 521–528https://doi.org/10.1002/cyto.a.24541
  38. [38]Din, N.U. and Yu, J. Training a deep learning model for single-cell segmentation without manual annotation. Sci Rep. 2021; 11: 23995https://doi.org/10.1038/s41598-021-03299-4
  39. [39]Ali, H.R., Jackson, H.W., Zanotelli, V.R.T., Danenberg, E., Fischer, J.R., Bardwell, H., Provenzano, E., CRUK IMAXT Grand Challenge Team, Rueda, O.M., Chin, S.-.F., Aparicio, S., Caldas, C., and Bodenmiller, B. Imaging mass cytometry and multiplatform genomics define the phenogenomic landscape of breast cancer. Nat Cancer. 2020; 1: 163–175https://doi.org/10.1038/s43018-020-0026-6
  40. [40]Legaz-García, M.D.C., Miñarro-Giménez, J.A., Menárguez-Tortosa, M., and Fernández-Breis, J.T. Generation of open biomedical datasets through ontology-driven transformation and integration processes. J Biomed Semantics. 2016; 7: 32https://doi.org/10.1186/s13326-016-0075-z
  41. [41]Adnan, M., Kalra, S., Cresswell, J.C., Taylor, G.W., and Tizhoosh, H.R. Federated learning and differential privacy for medical image analysis. Sci Rep. 2022; 12: 1953https://doi.org/10.1038/s41598-022-05539-7
  42. [42]Saldanha, O.L., Quirke, P., West, N.P., James, J.A., Loughrey, M.B., Grabsch, H.I., Salto-Tellez, M., Alwers, E., Cifci, D., Laleh, N.Ghaffari, Seibel, T., Gray, R., Hutchins, G.G.A., Brenner, H., van Treeck, M., Yuan, T., Brinker, T.J., Chang-Claude, J., Khader, F., Schuppert, A., Luedde, T., Trautwein, C., Muti, H.S., Foersch, S., Hoffmeister, M., Truhn, D., and Kather, J.N. Swarm learning for decentralized artificial intelligence in cancer histopathology. Nat Med. 2022; 28: 1232–1239https://doi.org/10.1038/s41591-022-01768-5
  43. [43]Warnat-Herresthal, S., Schultze, H., Shastry, K.L. et al. Swarm learning for decentralized and confidential clinical machine learning. Nature. 2021; 594: 265–270https://doi.org/10.1038/s41586-021-03583-3
  44. [44]Ma. J. Li, C., Shi, L., Ding, M., Wang, T., Han, Z., and Poor, H.V. When federated learning meets blockchain: a new distributed learning paradigm. IEEE Comput Intell Mag. 2022; 17: 26–33https://doi.org/10.1109/MCI.2022.3180932
  45. [45]Ahmedt-Aristizabal, D., Armin, M.A., Denman, S., Fookes, C., and Petersson, L. Graph-based deep learning for medical diagnosis and analysis: past, present and future. Sensors. 2021; 21: 4758https://doi.org/10.3390/s21144758
  46. [46]Y. Li, A. Gupta. Beyond grids: learning graph representations for visual recognition. In: S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, R. Garnett (editors), Advances in neural information processing systems 31, 2018. https://proceedings.neurips.cc/paper/2018/hash/4efb80f630ccecb2d3b9b2087b0f9c89-Abstract.html.
  47. [47]Failmezger, H., Muralidhar, S., Rullan, A., de Andrea, C.E., Sahai, E., and Yuan, Y. Topological tumor graphs: a graph-based spatial model to infer stromal recruitment for immunosuppression in melanoma histology. Cancer Res. 2020; 80: 1199–1209https://doi.org/10.1158/0008-5472.CAN-19-2268
  48. [48]Lu, C., Koyuncu, C., Corredor, G., Prasanna, P., Leo, P., Wang, X.X., Janowczyk, A., Bera, K., Lewis Jr, J., Velcheti, V., and Madabhushi, A. Feature-driven local cell graph (FLocK): new computational pathology-based descriptors for prognosis of lung cancer and HPV status of oropharyngeal cancers. Med Image Anal. 2021; 68: 101903https://doi.org/10.1016/j.media.2020.101903
  49. [49]Chuah, S. and Chew, V. High-dimensional immune-profiling in cancer: implications for immunotherapy. J Immunother Cancer. 2020; 8: e000363https://doi.org/10.1136/jitc-2019-000363
  50. [50]Biesecker, L.G. Hypothesis-generating research and predictive medicine. Genome Res. 2013; 23: 1051–1053https://doi.org/10.1101/gr.157826.113
  51. [51]Ou, F.-.S., Michiels, S., Shyr, Y., Adjei, A.A., and Oberg, A.L. Biomarker discovery and validation: statistical considerations. J Thorac Oncol. 2021; 16: 537–545https://doi.org/10.1016/j.jtho.2021.01.1616
  52. [52]Zhang, Z., Cui, P., and Zhu, W. Deep learning on graphs: a survey. IEEE Trans Knowl Data Eng. 2022; 34: 249–270https://doi.org/10.1109/TKDE.2020.2981333
  53. [53]N.G. Martin, S. Malacrino, M. Wojciechowska, L. Campo, H. Jones, D.C. Wedge, C. Holmes, K. Sirinukunwattana, H. Sailem, C. Verrill, J. Rittscher. A graph based neural network approach to immune profiling of multiplexed tissue samples. arXiv 2022;2202.00813. https://arxiv.org/abs/2202.00813.
  54. [54]Liu, C.C., Steen, C.B., and Newman, A.M. Computational approaches for characterizing the tumor immune microenvironment. Immunology. 2019; 158: 70–84https://doi.org/10.1111/imm.13101
  55. [55]Jiménez-Sánchez, D., Ariz, M., Chang, H., Matias-Guiu, X., de Andrea, C.E., and Ortiz-de-Solórzano, C. NaroNet: discovery of tumor microenvironment elements from highly multiplexed images. (])Med Image Anal. 2022; 78: 102384https://doi.org/10.1016/j.media.2022.102384

 

Linked Articles

Unknown widget #d2170c4d-a9cf-482f-ac17-ef77d57a1866

of type linkedContentList

Related Articles

Unknown widget #c2ffda61-8426-42f7-926b-03d7330eede2

of type relatedArticleListWidget