Skip Navigation
Skip to contents

Ann Coloproctol : Annals of Coloproctology

OPEN ACCESS
SEARCH
Search
Advanced Search
mobile menu button

Articles

Page Path
HOME > Ann Coloproctol > Volume 42(3); 2026 > Article
Original Article
Colorectal cancer
 From regression to machine learning: improving prediction of rectal cancer recurrence
Thanat Tantinam1,2,3,4orcid, Ekkarin Supatrakul2orcid, Pawit Sutharat2orcid, Suwan Sanmee2orcid, Kullawat Bhatanaprabhabhan3orcid, Boonchai Ngamsirimas3orcid, Nataphon Santrakul3orcid, Rangsima Thiengthiantham3orcid, Punnawat Chandrachamnong3orcid, Suradet Buakhrun3orcid, Sarawut Ramjan4orcid
Annals of Coloproctology 2026;42(3):293-302.
DOI: https://doi.org/10.3393/ac.2025.01067.0152
Published online: June 26, 2026

1Surgery Unit, Phatthalung Hospital, Phatthalung, Thailand

2Clinical Surgical Research Center, Chiang Mai University, Chiang Mai, Thailand

3Colorectal Surgery Unit, Faculty of Medicine Vajira Hospital, Navamindradhiraj University, Bangkok, Thailand

4College of Innovation, Thammasat University, Bangkok, Thailand

Correspondence to: Thanat Tantinam, MD Clinical Surgical Research Center, Chiang Mai University, 110 Intawaroros Rd, Si Phum, Muang, Chiang Mai 50200, Thailand Email: b.thanat@gmail.com
Co-correspondence to: Ekkarin Supatrakul, MD Clinical Surgical Research Center, Chiang Mai University, 110 Intawaroros Rd, Si Phum, Muang, Chiang Mai 50200, Thailand Email: ekkarin.sup@cmu.ac.th
• Received: September 5, 2025   • Revised: October 11, 2025   • Accepted: December 28, 2025

© 2026 The Korean Society of Coloproctology

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

prev next
  • 1,046 Views
  • 41 Download
Full Article

Download PDF Download PDF

  • Purpose
    Recurrence after curative-intent treatment of rectal cancer remains a major clinical challenge. Conventional regression models may not adequately capture complex clinical interactions, whereas machine learning algorithms may improve predictive accuracy. This study compared regression-based and machine learning models for predicting recurrence using routinely collected clinical data.
  • Methods
    A retrospective cohort of 581 patients with rectal adenocarcinoma treated between 2013 and 2022 at 2 university hospitals in Thailand was analyzed. Seventeen demographic, pathological, and treatment-related features were used to develop logistic regression (LR), random forest (RF), support vector machine, extreme gradient boosting (XGBoost), and light gradient boosting machine (LightGBM) models. Data were split using stratified train-test sampling (80:20). Fivefold cross-validation, calibration, and decision curve analyses were performed. Model performance was assessed using the area under the receiver operating characteristic curve (AUC), average precision (AP), F1 score, sensitivity, specificity, and feature importance.
  • Results
    All models demonstrated moderate discriminative performance. AUC values ranged from 0.626 (LightGBM) to 0.662 (XGBoost), while AP scores ranged from 0.483 (LightGBM) to 0.584 (RF). LR achieved perfect sensitivity (1.000) at the F1-optimized threshold, RF provided the strongest balance between precision and recall, and XGBoost achieved the highest AUC. Key predictors included carcinoembryonic antigen level, nodal status, surgical margin status, tumor grade, lymphovascular invasion, and the number of lymph nodes retrieved.
  • Conclusion
    Machine learning models modestly outperformed regression-based approaches, although their clinical value lies primarily in tailoring model strengths to specific priorities. Selecting LR, RF, or XGBoost according to objectives such as maximizing sensitivity, achieving balanced precision-recall, or enabling risk stratification may enhance recurrence surveillance in rectal cancer.
  • Keywords: Rectal cancer; Recurrence; Prediction method; Machine learning; Regression analysis
Rectal cancer is one of the most prevalent malignancies worldwide and constitutes a substantial burden in terms of morbidity, mortality, and healthcare costs [1]. It represents a major subset of colorectal cancers, with distinct anatomical, biological, and therapeutic considerations [2]. Despite advances in diagnosis, surgical techniques, chemoradiotherapy, and adjuvant treatments, disease recurrence remains a common and serious complication that significantly affects survival and quality of life [3]. Recurrent rectal cancer is associated with limited treatment options, increased complexity of care, and a poorer prognosis compared to primary disease.
Traditional statistical approaches, such as logistic regression (LR) and Cox proportional hazards models, have been widely used to identify risk factors for recurrence [4]. These models, while interpretable and methodologically robust, often fall short in capturing complex, nonlinear relationships within high-dimensional clinical datasets [5]. In recent years, data science and machine learning have emerged as promising tools to address these limitations by enabling more flexible and sophisticated analytical approaches.
The application of machine learning in healthcare has expanded rapidly, particularly in the field of oncology [6]. Machine learning algorithms can process large, diverse, and multidimensional datasets to detect patterns and generate predictions with potentially higher accuracy than conventional models [7–9]. In the context of rectal cancer, machine learning may offer a more effective means of identifying patients at increased risk of recurrence and guiding follow-up and management strategies accordingly [10].
Current predictive models for rectal cancer recurrence primarily rely on traditional statistical methods [11–13], which may not fully exploit the breadth of available clinical data or adequately capture complex interactions among risk factors. Moreover, these conventional models often demonstrate limited predictive accuracy and generalizability when applied in real-world clinical settings. Consequently, there is a critical need to develop more robust, adaptable, and clinically applicable tools to improve early identification of high-risk patients. Recent work by Jeon et al. [14] applied several machine learning algorithms, including LR, support vector machine (SVM), random forest (RF), and extreme gradient boosting (XGBoost), to predict recurrence after curative rectal cancer surgery. Their study demonstrated that machine learning methods can achieve high predictive performance in this context. However, most existing models, including that study, were developed using single-center datasets with a limited scope of variables. To extend this line of research, the present study uses a multicenter cohort and routinely collected clinical features to directly compare regression-based and machine learning approaches for recurrence prediction.
This study sought to address this gap by employing a data science–driven approach to identify appropriate models for predicting rectal cancer recurrence. By leveraging both classical statistical methods and multiple machine learning algorithms, the research aimed to compare their predictive performance and evaluate the feasibility of integrating machine learning into routine clinical practice.
Ethics statement
The study protocol was reviewed and approved by the institutional review boards of Vajira Hospital (No. COA 196/2568) and Maharaj Nakorn Chiang Mai Hospital (No. SUR-2568-0607). The requirement for written informed consent was waived due to the retrospective design and use of deidentified patient data. The study was conducted in compliance with the principles of the Declaration of Helsinki.
Study design and population
This study employed a retrospective cohort design using data from rectal cancer patients treated at 2 university hospitals in Thailand between January 2013 and December 2022. Eligible patients were those diagnosed with rectal adenocarcinoma and managed with curative intent, including surgical resection with or without neoadjuvant or adjuvant therapy. Patients treated with palliative intent, those with incomplete records, or those missing recurrence information were excluded. The overall study flow included case identification from institutional cancer registries, exclusion of ineligible cases, and subsequent data preprocessing for model development (Fig. 1).
Data collection and variables
Data collection included demographic and clinical characteristics at the time of diagnosis. Demographic variables comprised age, sex, body weight, height, comorbidities, smoking status, and family history of colorectal cancer. Clinical characteristics included tumor location, completeness of preoperative colonoscopy, findings from preoperative imaging, presenting symptoms, and preoperative carcinoembryonic antigen (CEA) levels. Treatment details were also documented, including the need for preoperative diversion ostomy, neoadjuvant treatment and its specific modalities, surgical intervention, pathological findings, adjuvant treatment, and completeness of both neoadjuvant and adjuvant therapies. The intention of curative treatment was also assessed. Treatment outcomes were recorded, including follow-up status (regular or lost to follow-up), survival status, date of last follow-up for surviving patients, recurrence status, and the type of recurrence, if present.
Tumor location was recorded as the distance from the anal verge in centimeters and classified into upper, middle, and lower rectum. A multidisciplinary team approach was used to confirm tumor location. Colonoscopy completeness was defined as successful cecal intubation prior to any treatment. Preoperative imaging data were extracted from radiology reports of computed tomography (CT) or magnetic resonance imaging scans. Preoperative CEA levels were documented prior to the initiation of treatment.
Neoadjuvant treatment was categorized as chemotherapy, chemoradiotherapy (CRT), or radiotherapy. Total neoadjuvant therapy (TNT) was also identified. The CRT protocols at the institutions included long-course radiotherapy with concurrent oral capecitabine or infused 5-fluorouracil/leucovorin (5-FU/LV), as well as short-course radiotherapy (SCRT; 5 Gy in 5 fractions). TNT protocols consisted of either induction chemotherapy followed by CRT or SCRT, or CRT or SCRT followed by consolidation chemotherapy. The specific treatment protocol for each patient was determined through multidisciplinary team discussions involving surgeons, medical oncologists, radiation oncologists, pathologists, and radiologists. Details of chemotherapy regimens, including the total number of cycles administered, were collected.
Completion of neoadjuvant or adjuvant treatment was defined as receipt of the full standard-dose chemotherapy regimen with completion of all planned cycles without postponement. The intention of curative treatment was assessed based on whether patients underwent all necessary treatments for both primary and metastatic disease with curative intent.
Follow-up adhered to the National Comprehensive Cancer Network (NCCN) guidelines, including CT imaging every 3 to 6 months for the first 2 years, followed by annual imaging, alongside CEA monitoring and scheduled colonoscopies. Loss to follow-up was defined as failure to attend scheduled hospital visits. Recurrence was classified as either local or metastatic. Recurrence confirmation was based on findings from surveillance imaging, endoscopic evaluation, or pathological verification, when applicable.
Data preprocessing
The initial dataset contained 96 variables. Data preparation followed a structured workflow:
(1) Data cleaning: Noninformative identifiers (hospital number, national ID) and duplicate records were removed. Categorical variables were standardized to ensure consistency.
(2) Handling missing values: Variables with more than 50% missing data were excluded (Supplementary Fig. 1), except for clinically relevant predictors such as incomplete neoadjuvant therapy and TNT, which were retained despite moderate missingness. Remaining missing values were imputed using multiple imputation by chained equations (MICE) with 5 imputations (m=5). Predictive mean matching was applied for continuous variables, and LR was used for categorical variables. Imputed datasets were combined using Rubin rules to obtain pooled parameter estimates, thereby preserving multivariable relationships and accounting for uncertainty due to missing data.
(3) Feature transformation: Ordinal encoding was applied to TNM stage and histologic grade. CEA was analyzed both as a continuous variable and dichotomized at 5 ng/mL. Margin status was recoded into a binary variable (negative vs. any positive).
(4) Class imbalance: As recurrence accounted for approximately 40% of cases, the SMOTETomek technique (synthetic minority oversampling combined with undersampling of the majority class) was applied within the training dataset to balance class distributions and improve minority-class detection.
Feature selection
Feature selection was performed using an integrated process combining clinical relevance, literature review, and statistical assessment of redundancy. Established prognostic factors for rectal cancer recurrence, including nodal status, margin status, tumor stage, and CEA level, were prioritized for inclusion based on prior evidence and expert consensus.
To minimize redundancy and ensure model stability, pairwise correlations between predictors were examined. Features exceeding commonly accepted thresholds for problematic multicollinearity (|r|>0.8) were considered candidates for exclusion, while clinically distinct variables were retained even in the presence of moderate correlations. This approach ensured that the final feature set balanced statistical robustness with clinical interpretability. The resulting predictor set encompassed demographic, tumor-related, pathological, and treatment-related variables considered most relevant for recurrence modeling. A detailed description of each feature is provided in Supplementary Table 1, and the correlation structure guiding variable inclusion is illustrated in Supplementary Fig. 2.
The primary outcome was disease recurrence, defined as reappearance of rectal cancer after curative treatment and classified as locoregional or metastatic recurrence.
Model development
Five predictive models were trained and compared: LR, RF, SVM with a radial basis function kernel (SVM-RBF), XGBoost, and light gradient boosting machine (LightGBM). Data were randomly split into training (80%) and test (20%) subsets using stratified sampling to preserve recurrence prevalence. Hyperparameter tuning was conducted using grid search with 5-fold cross-validation within the training set. Post-training probability calibration was applied using Platt scaling for linear models (LR, SVM) and isotonic regression for tree-based models (RF, XGBoost, LightGBM).
Model evaluation
Performance was assessed on the independent test set using multiple criteria: (1) area under the receiver operating characteristic (ROC) curve (AUC); (2) area under the precision-recall curve (average precision, AP); (3) F1 score; (4) sensitivity and specificity; (5) calibration plots; and (6) decision curve analysis. To enhance the reliability of performance estimates, each model underwent repeated stratified 5-fold cross-validation (5×10) to compute the mean±standard deviation for AUC and AP across 50 resamples. For the final held-out test set, 95% confidence intervals (CIs) for AUC and AP were estimated using nonparametric bootstrap resampling with 1,000 iterations. Model interpretability was addressed by permutation feature importance (LR, SVM), impurity-based importance (RF, XGBoost, LightGBM), and Shapley additive explanations (SHAP) values for tree-based models.
Statistical analysis
All data preparation, model development, and evaluation were conducted using Python ver. 3.7 (Python Software Foundation). Data manipulation and preprocessing were performed using Pandas and NumPy. Machine learning algorithms were implemented using the Scikit-learn package. The imbalanced-learn library was used to perform SMOTETomek resampling. Gradient boosting models were developed using both LightGBM and XGBoost. Model interpretability and feature importance were evaluated using SHAP. Visualization of data distributions and model outputs was performed using Matplotlib and Seaborn. LR with statistical inference was additionally conducted using the Statsmodels package. All code and model development were managed in PyCharm 2024.1.4, with version control maintained through Git to ensure full reproducibility.
Study cohort
From institutional databases, a total of 819 rectal cancer cases were identified during the study period. After exclusion of patients treated without curative intent, 581 patients were included in the final analysis (Fig. 1). This cohort was used to develop and validate both machine learning and regression models for recurrence prediction.
Baseline characteristics
Baseline demographic, clinicopathological, and treatment characteristics are summarized in Table 1. The median age was 60 years (interquartile range, 52–68 years), and 337 patients (58%) were male. Tumors were most commonly located in the lower rectum (46.1%) and were predominantly moderately differentiated adenocarcinomas (65.9%). Most patients presented with stage III disease (62.1%), followed by stage II (22.0%), stage IV (6.5%), and stage I (4.1%). Neoadjuvant therapy was administered to 332 patients (57.1%), including 74 (12.7%) who received TNT. Margin positivity was observed in 66 resections (11.4%), and lymphovascular invasion was present in 179 cases (30.8%). During follow-up, 235 patients (40.4%) developed recurrence, including both locoregional and distant failures. The median follow-up duration was 6.34 years. Among patients with M1 disease who underwent curative-intent resection, most achieved R0 status (91.7%), and recurrence patterns were consistent with those of the overall cohort, showing a predominance of distant over locoregional failures.
Model performance
Comparative performance of the 5 models in the independent test set is presented in Table 2. The AUC values ranged from 0.626 (95% CI, 0.520–0.731) for LightGBM to 0.662 (95% CI, 0.552–0.757) for XGBoost, with LR (AUC, 0.656; 95% CI, 0.553–0.752) and RF (AUC, 0.658; 95% CI, 0.552–0.757) demonstrating similar discrimination. LR achieved the highest sensitivity (1.000), identifying all recurrence cases at the selected threshold, although specificity was low (0.243). RF demonstrated the highest AP (0.584; 95% CI, 0.446–0.725), reflecting stronger precision-recall performance in the imbalanced dataset. Specificity was generally low across models, with LightGBM achieving the highest value (0.400). Calibration analysis showed slopes ranging from 0.173 to 0.439 and intercepts from −0.406 to −0.313, with Brier scores between 0.240 and 0.253. Performance metrics were derived from repeated stratified 5-fold cross-validation and bootstrapped 95% CIs based on 1,000 resamples of the test set. Model optimization was achieved through hyperparameter tuning (Supplementary Table 2), ensuring that each model was evaluated under its best-performing configuration.
A sensitivity analysis comparing models trained with and without SMOTETomek resampling is summarized in Supplementary Table 3. Across all algorithms, differences in AUC, AP, F1 score, sensitivity, specificity, and Brier score were minimal (ΔAUC range, −0.021 to 0.010; ΔAP range, −0.013 to 0.014). These findings indicate that class balancing had a negligible impact on model discrimination, precision-recall performance, or calibration, confirming the robustness of the results to imbalance adjustment.
ROC and decision curve analysis
Discriminative performance of the models is illustrated in Fig. 2A, which presents ROC curves for all 5 approaches. Although overall discrimination was modest across classifiers, XGBoost achieved the highest AUC (0.662), followed closely by RF (0.658) and LR (0.656). Clinical net benefit was assessed using decision curve analysis. Fig. 2B shows that all models provided greater net benefit than the “treat-all” or “treat-none” strategies across the clinically relevant threshold probability range of 10% to 30%. LR and RF demonstrated relatively consistent net benefit across thresholds, whereas XGBoost showed peak benefit at intermediate risk probabilities.
Threshold trade-off analysis
Threshold-specific performance metrics are summarized in Supplementary Table 4. Sensitivity, specificity, positive predictive value, and negative predictive value were evaluated at clinically relevant recurrence-risk thresholds of 10%, 20%, and 30%. Across models, lower thresholds (10%) yielded maximal sensitivity but poor specificity, whereas higher thresholds (30%) improved specificity at the cost of reduced sensitivity. These trade-offs allow clinicians to select decision thresholds aligned with clinical priorities, balancing recurrence detection against the risk of overtreatment.
Feature importance
Permutation-based and impurity-based feature importance analyses are shown in Supplementary Fig. 3. Across all 5 models, several predictors consistently ranked among the most influential. Pretreatment CEA level, pathological nodal status, margin status, patient age, tumor grade, and number of lymph nodes retrieved were repeatedly identified as high-contributing features. LR and SVM-RBF assigned relatively greater importance to treatment-related factors, including receipt and incompleteness of neoadjuvant therapy. In contrast, tree-based models such as RF, XGBoost, and LightGBM emphasized pathological and biological characteristics, with nodal status, CEA level, and margin status appearing prominently. Although the precise ranking of feature importance varied by algorithm, substantial overlap was observed among the top-ranked variables across models.
Model explainability with SHAP
SHAP summary plots illustrated the distribution, direction, and relative magnitude of feature contributions in the XGBoost and SVM-RBF models (Fig. 3). In both models, pretreatment CEA level and pathological nodal status exerted the largest influence on prediction output, with higher values shifting predictions toward recurrence. Margin status, tumor grade, patient age, and number of lymph nodes retrieved also demonstrated substantial contributions, although the magnitude of their effects varied between models. In XGBoost, margin positivity and moderate differentiation were particularly prominent, whereas in SVM-RBF, nodal status and CEA level dominated feature rankings. The plots further highlighted heterogeneity within individual variables, with some features exhibiting both positive and negative SHAP values depending on their specific levels or categories.
This study compared LR, RF, SVM-RBF, XGBoost, and LightGBM for predicting recurrence after curative-intent treatment of rectal cancer using routinely collected demographic, pathological, and treatment-related variables. All models demonstrated moderate discriminatory ability, with AUC values ranging from 0.626 to 0.662 and AP scores from 0.483 to 0.584. Despite modest ROC-based performance, several predictors consistently emerged as influential across models, including pretreatment CEA level, nodal status, margin status, tumor grade, lymphovascular invasion, and the number of lymph nodes retrieved. These variables align with established prognostic factors and underscore their continued relevance across both regression-based and machine learning approaches.
When benchmarked against prior literature, our models achieved lower ROC-based performance but comparable or superior AP. Xu et al. [15] reported AUCs of 0.734 to 0.761 in stage IV colorectal cancer patients, while several recent studies reported AUCs exceeding 0.75 for rectal cancer recurrence prediction [10, 14, 16]. Methodological and cohort differences likely explain these discrepancies. Xu et al. [15] analyzed a stage IV population with a recurrence rate of 78%, compared with approximately 40% in our cohort, and higher event prevalence is known to inflate AUC values. Moreover, prior studies incorporated perioperative and treatment-specific variables, such as blood loss, anesthesia duration, and chemotherapy regimens, which, although predictive, are not routinely retrievable from clinical databases [10, 14, 17]. In contrast, our study prioritized variables consistently recorded in hospital registries, thereby enhancing potential generalizability across diverse healthcare settings.
A distinct contribution of this study lies in explicitly mapping model strengths to specific clinical priorities. LR achieved perfect sensitivity at the F1-optimized threshold, ensuring that no recurrence cases were missed, albeit at the expense of poor specificity. RF provided the highest AP, offering the most favorable balance between precision and recall in an imbalanced dataset. XGBoost achieved the highest AUC, reflecting superior rank-ordering of recurrence risk with relatively higher specificity. Fig. 4 illustrates model selection according to different clinical goals. This framework shifts the emphasis from purely statistical performance toward actionable integration into clinical workflows.
Calibration analysis provided additional insight into the reliability of predicted probabilities for clinical application. Calibration slopes ranging from 0.17 to 0.44 and intercepts from −0.41 to −0.31 indicated moderate agreement between predicted and observed recurrence risk, while Brier scores between 0.24 and 0.25 reflected acceptable overall accuracy. Although none of the models achieved optimal calibration, these findings suggest that predicted probabilities were sufficiently consistent to support risk-stratified follow-up planning. Further recalibration using larger, prospective cohorts may improve reliability and facilitate integration into surveillance decision-making frameworks.
Threshold-based evaluation further supports the clinical adaptability of the proposed models. By reporting sensitivity, specificity, positive predictive value, and negative predictive value across recurrence-risk thresholds of 10% to 30%, this analysis demonstrates how operating points can be adjusted according to clinical priorities. Lower thresholds ensure maximal recurrence detection, whereas higher thresholds favor specificity and may reduce unnecessary surveillance intensity. Such threshold-guided interpretation enables individualized follow-up strategies aligned with available resources and patient risk tolerance.
The developed models are intended for use in the postoperative setting to assist with recurrence surveillance planning rather than for preoperative decision-making. Because several key predictors, including pathological nodal status, margin status, and lymphovascular invasion, become available only after surgical resection, the models are best applied during postoperative multidisciplinary reviews. In this context, they may help identify patients who could benefit from intensified surveillance or adjustment of adjuvant therapy. A conceptual flowchart illustrating this clinical workflow is provided in Fig. 5.
Several methodological strengths support the robustness of this work. Data were derived from 2 high-volume centers with standardized follow-up protocols, systematically preprocessed, and balanced using SMOTETomek. Sensitivity analyses comparing models trained with and without SMOTETomek confirmed that class balancing had minimal impact on discrimination, precision-recall performance, or calibration, supporting the stability of the findings. Missing data were addressed through multiple imputation, models were optimized via hyperparameter tuning, and interpretability was enhanced using permutation importance and SHAP analyses. Restricting predictors to routinely collected clinical variables further increases the feasibility of translation into real-world practice.
Nevertheless, several limitations should be acknowledged. The retrospective design and restriction to tertiary referral centers may limit generalizability to broader healthcare settings. Molecular and radiomic features were unavailable, constraining the predictive ceiling of the models. External validation was not feasible due to practical barriers related to data governance, privacy restrictions, and heterogeneity across institutional clinical databases. In particular, variation in variable definitions and data structures between hospital information systems limited integration of independent datasets. These challenges are consistent with those commonly reported in artificial intelligence-based rectal cancer research, where the absence of standardized multicenter datasets remains a major obstacle to external validation [18].
Although time-to-event analyses could provide valuable insight into recurrence timing and hazard dynamics, recurrence-date information in this retrospective cohort was incomplete and inconsistently recorded across institutions. Accordingly, the present study focused on binary prediction of recurrence to ensure analytic reliability and comparability across models. While this approach lacks temporal granularity, it remains clinically meaningful for identifying patients at elevated risk who may benefit from closer surveillance or tailored adjuvant management.
Future research should prioritize prospective study designs and time-dependent modeling approaches to estimate dynamic recurrence risk over clinically relevant intervals, such as 1-, 3-, and 5-year outcomes. Validation across diverse populations and external cohorts will be essential to confirm generalizability. Incorporation of molecular, genomic, and radiomic features may further enhance predictive performance. In addition, continuous-learning frameworks should be explored to maintain model accuracy as treatment strategies and patient characteristics evolve.
In conclusion, machine learning models modestly outperformed regression-based approaches for predicting rectal cancer recurrence, but their greatest value lies in aligning model use with specific clinical priorities. By matching model selection to goals of sensitivity, balance, or risk ranking, this work highlights a shift from maximizing performance metrics toward developing tools with practical clinical applicability.

Conflict of interest

No potential conflict of interest relevant to this article was reported.

Funding

This study was partially supported for publication by Chiang Mai University.

Author contributions

Conceptualization: TT, ES, PS, SS, SR; Data curation: TT; Formal analysis: TT; Funding acquisition: ****; Investigation: TT, ES, PS, SS, KB, BN, NS, RT, PC, SB; Methodology: TT, ES, PS, SS, SR; Supervision: ES, PS, SS, KB, BN, NS, RT, PC, SB, SR; Validation: SR; Visualization: TT; Writing–original draft: TT; Writing–review & editing: all authors. All authors read and approved the final manuscript.

Supplementary Table 1.

Predictor variables used for model development
ac-2025-01067-0152-Supplementary-Table-1.pdf

Supplementary Table 2.

Optimized hyperparameters for each predictive model
ac-2025-01067-0152-Supplementary-Table-2.pdf

Supplementary Table 3.

Comparison of model performance with and without SMOTETomek class balancing
ac-2025-01067-0152-Supplementary-Table-3.pdf

Supplementary Table 4.

Threshold-based performance metrics of the predictive models for recurrence after curative-intent rectal cancer treatment
ac-2025-01067-0152-Supplementary-Table-4.pdf

Supplementary Fig. 1.

Percentage of missing data by feature.
ac-2025-01067-0152-Supplementary-Fig-1.pdf

Supplementary Fig. 2.

Correlation matrix of numeric features.
ac-2025-01067-0152-Supplementary-Fig-2.pdf

Supplementary Fig. 3.

Feature importance analyses.
ac-2025-01067-0152-Supplementary-Fig-3.pdf
Supplementary materials are available from https://doi.org/10.3393/ac.2025.01067.0152.
Fig. 1.
Study flowchart. Rectal cancer cases were identified from institutional databases between 2013 and 2022. Patients treated without curative intent were excluded. The final cohort underwent data preprocessing, including data cleaning, transformation, and feature selection, prior to model development, training, and evaluation.
ac-2025-01067-0152f1.jpg
Fig. 2.
Analysis results for all 5 models: logistic regression (LR), random forest (RF), support vector machine with a radial basis function kernel (SVM-RBF), extreme gradient boosting (XGBoost), and light gradient boosting machine (LightGBM). (A) Receiver operating characteristic curves for all models in the test set. (B) Decision curve analysis illustrating the clinical net benefit of each model across clinically relevant threshold probabilities (10%–30% recurrence risk) compared with the default “treat-all” and “treat-none” strategies. AUC, area under the curve.
ac-2025-01067-0152f2.jpg
Fig. 3.
Shapley additive explanations (SHAP) summary plots showing feature contributions to prediction of recurrence in (A) extreme gradient boosting (XGBoost) and (B) support vector machine with a radial basis function kernel (SVM-RBF) models. A detailed description of each feature is provided in Supplementary Table 1. CEA, carcinoembryonic antigen; CRT, chemoradiotherapy.
ac-2025-01067-0152f3.jpg
Fig. 4.
Model selection guide based on clinical objectives. Flow diagram illustrating recommended model choice according to the primary goal: maximizing sensitivity (logistic regression, LR), achieving balanced precision-recall in imbalanced data (random forest, RF), or prioritizing specificity and risk stratification (extreme gradient boosting, XGBoost). AP, average precision; AUC, area under the curve.
ac-2025-01067-0152f4.jpg
Fig. 5.
Proposed clinical workflow for postoperative application of the recurrence prediction model. The model integrates pathologic and pretreatment clinical data to estimate recurrence probability, stratifying patients into illustrative risk categories (low, moderate, and high) based on model-derived probabilities. The blue box highlights the clinical decision point at which model output informs individualized surveillance intensity. aExample stratification thresholds derived from model-predicted recurrence probabilities.
ac-2025-01067-0152f5.jpg
Table 1.
Baseline demographic, clinicopathological, and treatment characteristics of patients with rectal cancer, stratified by recurrence status
Characteristic Total (n=581) Recurrence
No (n=346) Yes (n=235)
Age (yr) 60 (52–68) 61 (54–69) 60 (51–66)
No. of lymph nodesa 13 (7–18) 13 (7–18) 13 (7–18)
CEA level (ng/mL)b 5.4 (2.5–17.2) 3.8 (2.0–11.2) 10.0 (3.8–28.0)
Sex
 Male 337 (58.0) 195 (56.4) 142 (60.4)
 Female 244 (42.0) 151 (43.6) 93 (39.6)
Tumor stagec
 I 24 (4.1) 21 (6.1) 3 (1.3)
 II 128 (22.0) 87 (25.1) 41 (17.4)
 III 361 (62.1) 210 (60.7) 151 (64.3)
 IV 38 (6.5) 13 (3.8) 25 (10.6)
Tumor location
 Upper 105 (18.1) 70 (20.2) 35 (14.9)
 Middle 208 (35.8) 122 (35.3) 86 (36.6)
 Lower 268 (46.1) 154 (44.5) 114 (48.5)
Tumor grading
 Moderately differentiated 383 (65.9) 238 (68.8) 145 (61.7)
 Well differentiated 144 (24.8) 83 (24.0) 61 (26.0)
 Mucinous 25 (4.3) 15 (4.3) 10 (4.3)
 Poorly differentiated 15 (2.6) 7 (2.0) 8 (3.4)
 Signet 14 (2.4) 3 (0.9) 11 (4.7)
Neoadjuvant treatmentd
 No 243 (41.8) 160 (46.2) 83 (35.3)
 Chemoradiotherapy 305 (52.5) 178 (51.4) 127 (54.0)
 Chemotherapy 21 (3.6) 4 (1.2) 17 (7.2)
 Radiotherapy 6 (1.0) 2 (0.6) 4 (1.7)
Total neoadjuvant therapye
 No 254 (43.7) 147 (42.5) 107 (45.5)
 Yes 74 (12.7) 37 (10.7) 37 (15.7)
Postponement of neoadjuvant therapyf
 No 302 (52.0) 175 (50.6) 127 (54.0)
 Yes 26 (4.5) 9 (2.6) 17 (7.2)
Margin statusg
 Negative 515 (88.6) 327 (94.5) 188 (80.0)
 Positive 66 (11.4) 19 (5.5) 47 (20.0)
Lymphovascular invasionh
 No 240 (41.3) 183 (52.9) 57 (24.3)
 Yes 179 (30.8) 82 (23.7) 97 (41.3)

Values are presented as median (interquartile range) or number (%). Numbers may not sum to the total because of missing data.

CEA, carcinoembryonic antigen.

Missing data:

an=45 (7.7%);

bn=58 (10.0%);

cn=58 (10.0%);

dn=6 (1.0%);

en=14 (0.1%);

fn=253 (43.5%);

gn=26 (4.5%);

hn=162 (27.9%).

Table 2.
Comparative performance metrics for all models in the test set
Model AUC (95% CI)a APb (95% CI)a F1 score Sensitivity Specificity Calibration analysis Brier score
Slope Intercept
LR 0.656 (0.553–0.752) 0.528 (0.401–0.682) 0.639 1.000 0.243 0.173 –0.406 0.244
RF 0.658 (0.552–0.757) 0.584 (0.446–0.725) 0.593 0.979 0.114 0.415 –0.340 0.244
SVM-RBF 0.642 (0.533–0.741) 0.524 (0.392–0.681) 0.607 0.936 0.229 0.371 –0.343 0.253
XGBoost 0.662 (0.552–0.757) 0.527 (0.401–0.682) 0.623 0.915 0.314 0.439 –0.313 0.240
LightGBM 0.626 (0.520–0.731) 0.483 (0.370–0.638) 0.641 0.894 0.400 0.332 –0.335 0.250

AUC, area under the curve; CI, confidence interval; AP, average precision; LR, logistic regression; RF, random forest; SVM-RBF, support vector machine with a radial basis function kernel; XGBoost, extreme gradient boosting; LightGBM, light gradient boosting machine.

aValues derived from 1,000 bootstrap resamples of the test set.

bArea under the precision-recall curve.

  • 1. Siegel RL, Giaquinto AN, Jemal A. Cancer statistics, 2024. CA Cancer J Clin 2024;74:12–49. ArticlePubMed
  • 2. van der Sijp MP, Bastiaannet E, Mesker WE, van der Geest LG, Breugom AJ, Steup WH, et al. Differences between colon and rectal cancer in complications, short-term survival and recurrences. Int J Colorectal Dis 2016;31:1683–91. ArticlePubMedPMCPDF
  • 3. Kim HG, Kim HS, Yang SY, Han YD, Cho MS, Hur H, et al. Early recurrence after neoadjuvant chemoradiation therapy for locally advanced rectal cancer: characteristics and risk factors. Asian J Surg 2021;44:298–302. ArticlePubMed
  • 4. Lai IL, You JF, Chern YJ, Tsai WS, Chiang JM, Hsieh PS, et al. The risk factors of local recurrence and distant metastasis on pT1/T2N0 mid-low rectal cancer after total mesorectal excision. World J Surg Oncol 2021;19:116.ArticlePubMedPMCPDF
  • 5. Hua Y, Stead TS, George A, Ganti L. Clinical risk prediction with logistic regression: best practices, validation techniques, and applications in medical research. Acad Med Surg 2025 Mar 7 [Online]. https://doi.org/10.62186/001c.131964. Article
  • 6. Beam AL, Kohane IS. Big data and machine learning in health care. JAMA 2018;319:1317–8. ArticlePubMed
  • 7. Adlung L, Cohen Y, Mor U, Elinav E. Machine learning in clinical decision making. Med 2021;2:642–65. ArticlePubMed
  • 8. Shamout F, Zhu T, Clifton DA. Machine learning for clinical outcome prediction. IEEE Rev Biomed Eng 2021;14:116–26. ArticlePubMedPDF
  • 9. Weissler EH, Naumann T, Andersson T, Ranganath R, Elemento O, Luo Y, et al. The role of machine learning in clinical research: transforming the future of evidence generation. Trials 2021;22:537.ArticlePubMedPMCPDF
  • 10. Volovat CC, Scripcariu DV, Boboc D, Volovat SR, Vasilache IA, Ursulescu-Lupascu C, et al. Machine learning-based algorithms for enhanced prediction of local recurrence and metastasis in low rectal adenocarcinoma using imaging, surgical, and pathological data. Diagnostics (Basel) 2024;14:625.ArticlePubMedPMC
  • 11. Dai S, Ye Y, Kong X, Li J, Ding K. A predictive model for early recurrence of colorectal-cancer liver metastases based on clinical parameters. Gastroenterol Rep (Oxf) 2021;9:241–51. ArticlePubMedPMCPDF
  • 12. Hida K, Okamura R, Park SY, Nishigori T, Takahashi R, Kawada K, et al. A new prediction model for local recurrence after curative rectal cancer surgery: development and validation as an Asian collaborative study. Dis Colon Rectum 2017;60:1168–74. ArticlePubMed
  • 13. Yeo DM, Oh SN, Lee MA, Lee IK, Lee YS, Oh ST, et al. The development and validation of a predictive model for recurrence in rectal cancer based on radiological and clinicopathological data. Eur Radiol 2021;31:8586–96. ArticlePubMedPDF
  • 14. Jeon Y, Kim YJ, Jeon J, Nam KH, Hwang TS, Kim KG, et al. Machine learning based prediction of recurrence after curative resection for rectal cancer. PLoS One 2023;18:e0290141. ArticlePubMedPMC
  • 15. Xu Y, Ju L, Tong J, Zhou CM, Yang JJ. Machine learning algorithms for predicting the recurrence of stage IV colorectal cancer after tumor resection. Sci Rep 2020;10:2519.ArticlePubMedPMCPDF
  • 16. Zhao B, Gabriel RA, Vaida F, Lopez NE, Eisenstein S, Clary BM. Predicting overall survival in patients with metastatic rectal cancer: a machine learning approach. J Gastrointest Surg 2020;24:1165–72. ArticlePubMedPDF
  • 17. Reddavid R, Elmore U, Moro J, De Nardi P, Biondi A, Persiani R, et al. Dynamic prediction of rectal cancer relapse and mortality using a landmarking-based machine learning model: a multicenter retrospective study from the Italian Society of Surgical Oncology-Colorectal Cancer Network Collaborative Group. Cancers (Basel) 2025;17:1294.ArticlePubMedPMC
  • 18. Kim M, Park T, Oh BY, Kim MJ, Cho BJ, Son IT. Performance reporting design in artificial intelligence studies using image-based TNM staging and prognostic parameters in rectal cancer: a systematic review. Ann Coloproctol 2024;40:13–26. ArticlePubMedPMCPDF

Figure & Data

References

    Citations

    Citations to this article as recorded by  

      • Cite this Article
        Cite this Article
        export Copy Download
        Close
        Download Citation
        Download a citation file in RIS format that can be imported by all major citation management software, including EndNote, ProCite, RefWorks, and Reference Manager.
        Format:
        • RIS — For EndNote, ProCite, RefWorks, and most other reference management software
        • BibTeX — For JabRef, BibDesk, and other BibTeX-specific software
        Include:
        • Citation for the content below
        From regression to machine learning: improving prediction of rectal cancer recurrence
        Ann Coloproctol. 2026;42(3):293-302.   Published online June 26, 2026
        DOI: https://doi.org/10.3393/ac.2025.01067.0152
        Close
      • XML DownloadXML Download
      Figure
      • 0
      • 1
      • 2
      • 3
      • 4
      From regression to machine learning: improving prediction of rectal cancer recurrence
      Image Image Image Image Image
      Fig. 1. Study flowchart. Rectal cancer cases were identified from institutional databases between 2013 and 2022. Patients treated without curative intent were excluded. The final cohort underwent data preprocessing, including data cleaning, transformation, and feature selection, prior to model development, training, and evaluation.
      Fig. 2. Analysis results for all 5 models: logistic regression (LR), random forest (RF), support vector machine with a radial basis function kernel (SVM-RBF), extreme gradient boosting (XGBoost), and light gradient boosting machine (LightGBM). (A) Receiver operating characteristic curves for all models in the test set. (B) Decision curve analysis illustrating the clinical net benefit of each model across clinically relevant threshold probabilities (10%–30% recurrence risk) compared with the default “treat-all” and “treat-none” strategies. AUC, area under the curve.
      Fig. 3. Shapley additive explanations (SHAP) summary plots showing feature contributions to prediction of recurrence in (A) extreme gradient boosting (XGBoost) and (B) support vector machine with a radial basis function kernel (SVM-RBF) models. A detailed description of each feature is provided in Supplementary Table 1. CEA, carcinoembryonic antigen; CRT, chemoradiotherapy.
      Fig. 4. Model selection guide based on clinical objectives. Flow diagram illustrating recommended model choice according to the primary goal: maximizing sensitivity (logistic regression, LR), achieving balanced precision-recall in imbalanced data (random forest, RF), or prioritizing specificity and risk stratification (extreme gradient boosting, XGBoost). AP, average precision; AUC, area under the curve.
      Fig. 5. Proposed clinical workflow for postoperative application of the recurrence prediction model. The model integrates pathologic and pretreatment clinical data to estimate recurrence probability, stratifying patients into illustrative risk categories (low, moderate, and high) based on model-derived probabilities. The blue box highlights the clinical decision point at which model output informs individualized surveillance intensity. aExample stratification thresholds derived from model-predicted recurrence probabilities.

      Fig. 1.

      Fig. 2.

      Fig. 3.

      Fig. 4.

      Fig. 5.

      From regression to machine learning: improving prediction of rectal cancer recurrence
      Characteristic Total (n=581) Recurrence
      No (n=346) Yes (n=235)
      Age (yr) 60 (52–68) 61 (54–69) 60 (51–66)
      No. of lymph nodesa 13 (7–18) 13 (7–18) 13 (7–18)
      CEA level (ng/mL)b 5.4 (2.5–17.2) 3.8 (2.0–11.2) 10.0 (3.8–28.0)
      Sex
       Male 337 (58.0) 195 (56.4) 142 (60.4)
       Female 244 (42.0) 151 (43.6) 93 (39.6)
      Tumor stagec
       I 24 (4.1) 21 (6.1) 3 (1.3)
       II 128 (22.0) 87 (25.1) 41 (17.4)
       III 361 (62.1) 210 (60.7) 151 (64.3)
       IV 38 (6.5) 13 (3.8) 25 (10.6)
      Tumor location
       Upper 105 (18.1) 70 (20.2) 35 (14.9)
       Middle 208 (35.8) 122 (35.3) 86 (36.6)
       Lower 268 (46.1) 154 (44.5) 114 (48.5)
      Tumor grading
       Moderately differentiated 383 (65.9) 238 (68.8) 145 (61.7)
       Well differentiated 144 (24.8) 83 (24.0) 61 (26.0)
       Mucinous 25 (4.3) 15 (4.3) 10 (4.3)
       Poorly differentiated 15 (2.6) 7 (2.0) 8 (3.4)
       Signet 14 (2.4) 3 (0.9) 11 (4.7)
      Neoadjuvant treatmentd
       No 243 (41.8) 160 (46.2) 83 (35.3)
       Chemoradiotherapy 305 (52.5) 178 (51.4) 127 (54.0)
       Chemotherapy 21 (3.6) 4 (1.2) 17 (7.2)
       Radiotherapy 6 (1.0) 2 (0.6) 4 (1.7)
      Total neoadjuvant therapye
       No 254 (43.7) 147 (42.5) 107 (45.5)
       Yes 74 (12.7) 37 (10.7) 37 (15.7)
      Postponement of neoadjuvant therapyf
       No 302 (52.0) 175 (50.6) 127 (54.0)
       Yes 26 (4.5) 9 (2.6) 17 (7.2)
      Margin statusg
       Negative 515 (88.6) 327 (94.5) 188 (80.0)
       Positive 66 (11.4) 19 (5.5) 47 (20.0)
      Lymphovascular invasionh
       No 240 (41.3) 183 (52.9) 57 (24.3)
       Yes 179 (30.8) 82 (23.7) 97 (41.3)
      Model AUC (95% CI)a APb (95% CI)a F1 score Sensitivity Specificity Calibration analysis Brier score
      Slope Intercept
      LR 0.656 (0.553–0.752) 0.528 (0.401–0.682) 0.639 1.000 0.243 0.173 –0.406 0.244
      RF 0.658 (0.552–0.757) 0.584 (0.446–0.725) 0.593 0.979 0.114 0.415 –0.340 0.244
      SVM-RBF 0.642 (0.533–0.741) 0.524 (0.392–0.681) 0.607 0.936 0.229 0.371 –0.343 0.253
      XGBoost 0.662 (0.552–0.757) 0.527 (0.401–0.682) 0.623 0.915 0.314 0.439 –0.313 0.240
      LightGBM 0.626 (0.520–0.731) 0.483 (0.370–0.638) 0.641 0.894 0.400 0.332 –0.335 0.250
      Table 1. Baseline demographic, clinicopathological, and treatment characteristics of patients with rectal cancer, stratified by recurrence status

      Values are presented as median (interquartile range) or number (%). Numbers may not sum to the total because of missing data.

      CEA, carcinoembryonic antigen.

      Missing data:

      n=45 (7.7%);

      n=58 (10.0%);

      n=58 (10.0%);

      n=6 (1.0%);

      n=14 (0.1%);

      n=253 (43.5%);

      n=26 (4.5%);

      n=162 (27.9%).

      Table 2. Comparative performance metrics for all models in the test set

      AUC, area under the curve; CI, confidence interval; AP, average precision; LR, logistic regression; RF, random forest; SVM-RBF, support vector machine with a radial basis function kernel; XGBoost, extreme gradient boosting; LightGBM, light gradient boosting machine.

      Values derived from 1,000 bootstrap resamples of the test set.

      Area under the precision-recall curve.

      Table 1.

      Table 2.

      Ann Coloproctol : Annals of Coloproctology Twitter Facebook
      © Korean Society of Coloproctology.
      TOP