Subject Subtyping Skill

Overview

subject-subtyping discovers unsupervised subject groups from tabular imaging or multimodal features. It exports subtype assignments, latent embeddings, silhouette diagnostics, and a fitted estimator checkpoint. The reporting route also exports patient-level QA, feature profiles, group sizes and figures.

Research use only: clusters are not clinical diagnoses. The legacy training checkpoint does not bundle the training imputer/scaler and cannot by itself assign new patients from raw features. Preserve the fitted preprocessing and feature order in a separately validated deployment pipeline before future-patient inference; spectral/consensus training does not define an out-of-sample rule.

Supported models

Model	Method	Typical use
kmeans	Euclidean partitioning	compact baseline
gmm	Gaussian mixture	soft distributional subtypes
spectral	graph spectral clustering	non-convex structure
nmf	non-negative embedding + K-means	parts-based phenotypes
consensus	bootstrap co-clustering	stability-focused analysis
pca	PCA embedding + K-means	linear latent subtypes
autoencoder	neural embedding + K-means	nonlinear latent subtypes

Outcome labels must not be used to choose the number of clusters. Clinical outcomes may be tested only after subtype definitions are frozen.

---

Installation

pip install numpy pandas scipy scikit-learn joblib torch

Verify:

python -c "import sklearn, torch; print('Subtyping models OK')"

---

Workflows

1. Prepare features

Input is a CSV with subject_id and numeric features. Do not include diagnosis, survival, or treatment outcome columns among the clustering features.

subject_id,roi_001,roi_002,network_fc,brain_age_gap
sub-001,0.12,-0.04,0.31,2.1
sub-002,0.08,-0.09,0.27,-1.4

2. Consensus clustering

python skills/subject-subtyping/scripts/train_reference.py \
  --features features.csv \
  --subject-col subject_id \
  --model consensus \
  --n-clusters 3 \
  --seed 123 \
  --output-dir run_models_output/subtyping_consensus

3. PCA or autoencoder embeddings

python skills/subject-subtyping/scripts/train_reference.py \
  --features features.csv \
  --model pca \
  --n-clusters 4 \
  --latent-dim 8 \
  --output-dir run_models_output/subtyping_pca

python skills/subject-subtyping/scripts/train_reference.py \
  --features features.csv \
  --model autoencoder \
  --n-clusters 4 \
  --latent-dim 8 \
  --epochs 200 \
  --output-dir run_models_output/subtyping_ae

4. Select the number of clusters

Run a prespecified range such as k=2..8, compare silhouette and bootstrap stability, then freeze k before association with clinical endpoints. Repeat the final model across seeds when cluster stability is central to the claim.

5. Export the requested patient outputs

Use frozen assignments from the training route above, or an existing study; do not retrain just to produce a report. Provide feature_columns.json, an explicit JSON array of the prespecified imaging/multimodal features. Exclude IDs, site identifiers and outcome/diagnosis variables from the feature set. Input feature IDs must be unique, and assignments must have exactly the same subject set; the report joins by ID, never by assumed row order.

python skills/subject-subtyping/scripts/report_subtypes.py \
  --features features.csv --feature-columns feature_columns.json \
  --assignments run_models_output/subtyping_consensus/predictions.csv \
  --output-dir run_models_output/subtyping_report

Outputs include subtype_assignments.csv, subtype_profiles.csv, subtype_counts.csv, embedding.csv, subtype_report.png, report.md, qc.json, descriptive preprocessing parameters and a verified manifest. Profiles retain observed-value missingness. Standardization and fallback PCA are descriptive, not a new trained subtype definition or evidence of replication. There are no fabricated outcome associations or subtype confidence probabilities.

If completed resampling/seed runs exist, repeat --replicate predictions_seed2.csv to add permutation-invariant adjusted Rand and optimally matched cluster Jaccard scores in stability.csv. These runs must cover the same subjects and genuinely represent the specified perturbation. Without them, stability is explicitly not_evaluated; request/scopingly plan resampling if the user requires it.

python -m models.common.research_outputs run_models_output/subtyping_report/run_manifest.json \
  --require subtype_assignments subtype_profiles subtype_counts subtype_figure report

Use a new output directory for every run. With requested stability, add --require stability to the final check. An absent requested output is incomplete work, not an optional artifact silently omitted after training.

---

Input / Output Summary

Item	Format
Input	CSV; one row per subject
Required	numeric feature columns
Optional	configurable subject ID column
Assignments	predictions.csv with subject and subtype
Embedding	latent dimensions in predictions.csv
Metrics	metrics.json including silhouette and cluster count
Model	checkpoint.joblib
Provenance	config.json, run_manifest.json

---

Testing

pytest models/tests/test_extended_models.py -q
python skills/subject-subtyping/scripts/train_reference.py --help
pytest models/tests/test_research_output_skills.py -q

---

Directory Reference

models/subtyping/
├── estimators.py       clustering and embedding implementations
└── train.py            artifact-producing CLI

skills/subject-subtyping/
├── SKILL.md
└── scripts/
    ├── train_reference.py
    └── report_subtypes.py

---

Reference

• Consensus clustering uses bootstrap co-assignment frequencies.
• NMF inputs are transformed to a non-negative scale before decomposition.
• PCA and autoencoder modes cluster the learned embedding rather than raw data.

---

Created At: 2026-07-26 HKT Last Updated At: 2026-09-14 15:05:31.978 HKT Author: chengwang96