Learning

Metastasis prediction using machine learning

Classify liver-metastasis status in a retrospective colorectal cancer cohort

A Neural Designer network combines clinical, tumour and targeted-sequencing descriptors from a filtered MSK-MET table. On 707 internally held-out rows it reaches ROC AUC 0.856 and 78.4% accuracy. Because several inputs describe metastatic burden or later outcomes, this is a retrospective research classifier—not an early-risk, diagnostic or treatment model.

0.856testing ROC AUC (95% CI 0.833–0.878)

78.4%testing accuracy at threshold 0.50

3,537unique sample rows

525–7–1final encoded network

Review the testing evidence Download the data

Scientific objective
Data and provenance
Model
Training
Selection
Validation
Inference
Validity

1. Scientific objective

The target records whether a colorectal cancer sample is labelled with distant liver metastasis in the derived table. The defensible objective is to reproduce associations and classify this recorded status inside the available cohort. It is not to forecast which metastasis-free patient will later develop liver disease.

Retrospective phenotyping

Prioritize or quality-check cohort rows using a reproducible multivariable score.

Organotropism research

Study how clinical burden and genomic descriptors co-vary with recorded liver involvement.

Reproducible benchmarking

Inspect the split, preprocessing, selected architecture, exact weights and operating thresholds.

Computational oncologyCancer genomicsBiostatisticsTranslational researchClinical data science

Scope. The model is an internally evaluated research example derived from one institutional cohort. It does not establish temporal prediction, clinical utility, calibration in another population or benefit to patients.

2. Data and provenance

The table contains 3,537 unique sample identifiers from colorectal cancer records derived from the public MSK-MET study. It has 510 columns: one identifier, 492 mutation-count fields, 16 other clinical or tumour descriptors and the binary target distant_metastasis_liver.

Subset	Rows	Liver metastasis: Yes	No	Purpose
Training	2,123	1,185	938	Estimate model parameters
Selection	707	396	311	Choose the hidden-layer size
Testing	707	437	270	Final internal evaluation
Total	3,537	2,018 (57.1%)	1,519 (42.9%)	Derived analysis cohort

Distribution of liver-metastasis yes and no labels — The target is moderately imbalanced: 57.1% of rows carry the liver-metastasis label.

Largest univariate correlations with the liver-metastasis label — The two largest associations are metastatic-burden variables. These coefficients are descriptive, not causal importance estimates or prospective biomarkers.

Model input contract

Eleven mutation fields are constant in this subset and are marked unused. The remaining 497 raw inputs comprise 481 mutation counts and 16 clinical or tumour fields. Categorical expansion produces 525 numeric model features. The source CSV uses NA in five fields; the downloadable converter reproduces the project’s numeric mean imputation and categorical missing-value encoding.

Temporal leakage boundary. metastasis_count, metastasis_primary_site_count, age_at_first_metastasis_diagnostic and mortality_3_years are not clean baseline predictors for a pre-metastasis decision. Their inclusion makes the endpoint contemporaneous or retrospective.

Provenance. The source is the MSK-MET clinico-genomic resource reported by Nguyen et al. The full study contains more than 25,000 patients across 50 cancer types; this tutorial uses a filtered 3,537-row colorectal table and must not be described as validation on the full cohort.

3. Model

The starting classifier scales 525 numeric features created from 497 raw inputs and connects them directly to one sigmoid output. With no hidden layer, the initial model contains 526 trainable parameters and provides the baseline for training and hidden-neuron selection.

Output contract. Larger scores rank rows toward the supplied Yes label. The sigmoid score has not been independently calibrated as an individual probability of liver metastasis.

Initial liver-metastasis classifier with 497 raw inputs, 525 encoded features and one output — Initial direct 525–1 encoded architecture used before growing-neurons selection.

4. Training strategy

The stored training strategy minimizes class-weighted squared error with the quasi-Newton method and L2 regularization weight 0.01. The positive and negative class weights are 0.8764 and 1.1643, respectively.

Training error falls from 1.112 to 0.439 in the displayed run. Selection error reaches its lowest region early—about 0.664—and then increases to roughly 0.747 while training loss continues to decline. This is evidence of overfitting in the initial run; it is not “strong generalization.”

Training and selection errors for the initial liver-metastasis network — The divergence between training and selection errors motivates capacity control and independent testing.

5. Model selection and baseline

Growing-neurons selection evaluates one to ten hidden tanh neurons with three trials per size. The reported minimum selection error is 0.6297 at seven neurons, with training error 0.1301.

Liver-metastasis training and selection errors by hidden-neuron count — Selection error is nearly flat across the search (approximately 0.630–0.646), so seven neurons are the stored choice, not evidence of a uniquely superior architecture.

Selected architecture

The final model expands the hidden layer to seven tanh neurons and retains one sigmoid output. The encoded 525–7–1 network contains 3,690 trainable parameters and is the version used for the ROC curve, confusion analysis and Python export.

Selected liver-metastasis network with 525 encoded inputs, seven hidden neurons and one output — Final 525–7–1 architecture obtained after growing-neurons selection.

Reference	Testing accuracy	Interpretation
Majority-class baseline	61.8%	Label every testing row as liver-metastasis positive
Final neural network	78.4%	16.6 percentage points above the internal baseline

6. Scientific validation

Final performance is calculated once on the 707 testing rows. Neural Designer reports ROC AUC 0.856 with a 95% confidence interval of 0.833–0.878. The regenerated ROC graphic marks threshold 0.51, the testing point nearest the upper-left corner.

ROC curve for the 707 liver-metastasis testing rows — Testing ROC curve. The marked point is selected from this internal test curve and is descriptive; it is not an externally validated clinical cutoff.

Actual / predicted at 0.50	Positive	Negative	Total
Liver metastasis: Yes	344	93	437
Liver metastasis: No	60	210	270
Total	404	303	707

Testing metric	Value	Reading
Accuracy	78.4%	554 of 707 rows are classified correctly
Precision	85.1%	344 of 404 positive calls match the supplied label
Sensitivity	78.7%	344 of 437 labelled positives are detected
Specificity	77.8%	210 of 270 labelled negatives are rejected
F1 score	0.818	Harmonic balance of precision and sensitivity
Balanced accuracy	78.2%	Mean of sensitivity and specificity
ROC AUC	0.856	Ranking discrimination across all thresholds

Interpretation. The network discriminates the recorded endpoint better than the majority baseline, but 93 testing positives are missed and 60 negatives are flagged at 0.50. Accuracy and AUC do not establish calibration, net benefit, temporal validity or clinical utility.

7. Inference and reproducibility

The appropriate deployment for this example is a reproducible batch-research workflow, not a patient-facing calculator. Manual entry of 497 raw variables would be error-prone, and the endpoint is not defined for prospective clinical use.

Versioned MSK-MET-format extract

Schema and category checks

497-to-525 encoding

525–7–1 exported network

Score and declared threshold

Aggregate research report and expert review

Illustrative threshold policies

These scenarios are recalculated from the 707 testing rows with the exact Python export. They demonstrate the trade-off only; a real operating point requires a prespecified use case and separate external evaluation.

Retrospective policy	Threshold	Sensitivity	Specificity	False negatives	False positives	Research use
Sensitivity-first review	0.30	87.6%	62.6%	54	101	Reduce missed labelled rows at the cost of more manual review
Nearest ROC corner	0.51	78.5%	78.9%	94	57	Balanced internal discrimination
Specificity-first review	0.70	65.7%	85.9%	150	38	Fewer false flags, substantially more missed positives

Not clinical thresholds. All three rows reuse the same internal testing set. They are unsuitable for diagnosis, surveillance intervals, treatment selection or patient counselling.

Run the exact model in Python

The package accepts the raw semicolon-delimited table, reproduces the project’s categorical expansion and missing-value handling, and appends a score and threshold label to every row. It contains the model definition and schema, but no patient-level records.

python score_csv.py liver_metastasis.csv scored_rows.csv --threshold 0.50

Reproduce the calculation

The Python ZIP contains the exact exported weights, a 525-feature schema, the validated raw-to-feature converter and a batch scoring script. The patient-level dataset is not duplicated inside the package.

Download Python model (ZIP)Download the existing research dataset (CSV)

8. Validity, uncertainty and limitations

Retrospective endpoint. The label records known liver involvement. Several inputs describe metastatic burden or later outcomes, so the model does not answer a clean pre-metastasis prediction question.
No declared index date. Predictor availability is not frozen at a common clinical decision point. A prospective version must define time zero and retain only information available then.
Internal random split only. The 2,123/707/707 partition evaluates rows from the same derived institutional cohort. There is no temporal, geographic or external-centre validation.
Selection uncertainty. Hidden-layer selection errors are nearly flat, and architecture choice is based on one selection split. Repeated grouped resampling is needed to quantify stability.
Calibration unverified. The sigmoid score is useful for ranking and threshold demonstrations, but no calibration curve, calibration slope/intercept or external recalibration is reported.
Prevalence dependence. The testing prevalence is 61.8%; precision and negative predictive value will change in populations with different case mix and endpoint ascertainment.
Feature and coding drift. Sequencing panel versions, variant processing, tumour purity, missingness and category definitions must remain compatible with the training pipeline.
Equity and subgroup performance untested. Aggregate results do not establish comparable performance by sex, race category, age, primary site or molecular subtype.
No clinical-impact evaluation. There is no decision-curve analysis, prospective workflow study, comparison with standard care or evidence that model use improves outcomes.

Decision boundary. Use this package for education, reproducibility and retrospective cohort research. Do not use it to diagnose liver metastasis, set surveillance schedules, select therapy or advise an individual patient.

What a prospective redevelopment would require

Define a baseline decision time; exclude downstream variables such as known metastatic counts and three-year mortality; specify eligible patients and outcome ascertainment; validate by patient, time and external centre; assess calibration and clinical utility; and report the study using TRIPOD+AI with risk of bias assessed using PROBAST+AI.

References

Nguyen B, Fong C, Luthra A, et al. Genomic characterization of metastatic patterns from prospective clinical sequencing of 25,000 patients. Cell. 2022;185(3):563–575.e11.
MSK MetTropism (MSK-MET) study. cBioPortal for Cancer Genomics.
Collins GS, Moons KGM, Dhiman P, et al. TRIPOD+AI statement. BMJ. 2024;385:e078378.
Moons KGM, Damen JAAG, Kaul T, et al. PROBAST+AI. BMJ. 2025;388:e082505.